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FOREWORD 



JAMES L. DENNIS. M.D.* 
Universiiy of OWahoma Medical Center 



Distinguished guests and colleagues, wcleomc to Oklahoma, to Oklahoma 
City and to the University of Oklahoma Medieal Center eampus. 

Someone has described planning as the projection of a wedding between 
the ideal and the practical. Tliis is not easily accomplished in our complex society, 
but one of the catalytie agents tliat can make such a wedding possible is computer 
science. Cuntnt pressures upon medical education arc twofold. From one side, 
there is the explosion of knowlctTgc and from the other side, the pressures from 
society to fill the gaps in the health care system and to maintain the ongoing 
quality of the physicians we graduate to prevent obsolescence. Computer science 
offers perhaps the most altractivo possibility for bringing order out of cl^aos. 

When faced with the problem of relating medical cdticallon to societal needs, 
one must bo very concerned about the protection of academic excellence and 
its future enhancement. The pressures to become involved In the delivery of 
mass health scn'iccs to relieve the shortages and deficits that exist in the nation 
are tremendous. Many people now l\jm to the medical centers to solve their 
problems and vn'c cannot Ignore o\jr rcs|X)nsibililIos. Ilic silualion is similar to 
that of the chicken and the pig who were worried about their kind master who 
was has Ing a very liard lime. Th^' wondered what they covild do to belj). Tlic 
chicken said, can feed him,*^ and the pig asked, *Fccd him what ? The 
chick^'n responded, *'\Vc can feed liim ham and eggs'* to which the pig rcaelod 
immediately, *'For yovi that's only a commitmciU, but for me that's total involve- 
ment." Certainly, anytime a mrxlic'al educational institution takes on the re- 
sponsibility of an undcfincxl mass health service program, this means tola! In- 
volvement and with the same results that the pig antieioaled. Therefore, we 
must have a protected core of academic excellence, as is found In the research 
and cduealionalpro^ams. Here is the fountainhead for new knowledge and new 
developments. The development of a health center concept who.^c aelivilies get 
programming closer to the public, provides communication from the research 
Dcmh and the academic desk by way of programs that permit an Immediate 
Iran s mission of nesv knowledge and new technology to the public. 

We thought that the health scrocc problems on the one hand and the health 
icicDCC challenges on the other were more than a single medical Institution in a 
stale such as Oklahoma could meet, without forming an alliance with others In 
our community who have responsibilities for the bealtn sc Knees. We went to out- 
standing citizens of this stale and discs jsed the possibility of developing what 
we now call the Health Sciences Foundation. This foundation, was formed not 

riexidenl for MedJrti/ Center Affein ond DeiJn, ScVx>/ of Medicine 



to run the proposed Oklahoma Health Center but to aid and assist institutions 
who were willing to bui!d in geographic proximity to the University as a core 
and to commit themselves to assisting the University in its mission of health man- 
power production and the p\irsuit of knowledge. The planning is well along 
and buildings are under construc tion. The Health Center \n 111 have interfaces with 
institutions at city, county, state, and Federal health ngency levels and with 
both private and public agencies and institutions, ^^any central services and 
facilities >vill be shared by all of these institutions. This means that funds for 
capital inveshnents that we receive at the Medical Center can go into labora- 
tories, classrooms and beds rather than into steam, refrigeration and laundries. 
The Oklahoma Health Center is coordinated by an operations committee com- 
prised of the heads of each of (he institutions (hat \WlI locate here. This commit- 
tee delennincs feasibility and recommends policy and procedures to our In- 
dividual governing boaras. These are presented to the Foundation for implemen- 
tation. All of these activities arc tied together by a rather ambitious computer 
P’^ogrom and, of course, the concept of reaching out into the stale is very much 
dependent upon computer technology and computer-based communication. 

This conference, to explore the role of computers in medical education, is a 
very timely one because of the current examination into curricula, programs and 
facilities for health manpower. KJuch should be learned from this opporhinity for 
knowledgeable people to interact, and the discussion could bav. a signifii-ant 
Impact on the future of medical education. 
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MEDICAL TECHNOLOGY AND SOCIAL CHANGE 

J. HERBERT HOLLOMON, PH D.* 



Good morning, ladies and gentlemen. 
Welcome to the conference on computer 
and medical technology. 

I speak to you on a subject on wliich I 
am not an expert. So I would like to 
make the subject somewhat more gener* 
al at first. I wish to discuus the general 
relationship between technology and so- 
cial change. We face a time in nistoiy in 
which many people believo that technol- 
ogy and its consequences have become 
so harmful to the society that this harm 
overrides the possible good that le< hnol- 
ogy brings. People — )Oung people par- 
ticularly, perhaps more perceptive, more 
caring, more affluent than any young 
people of any society ~ have begun to 
ask questions: “How is it that you allow 
technology to pollute the atmoq>here, 
when you build factories? How is it that 
the people who utilize technology allow 
the streams to be polluted so that our 
water supplies arc undrinkable?" 'How 
is it that a society can allow a system to 
operate that allows 50,000 people to be 
killed in highly technical devices called 
automobiles, and 2 or 3 or 4 million, 
whatever the appropriate ntmber, 
wounded?* "How can a society net con- 
sider deeply the problems that are creat- 
ed uilb the introduction of agrlciillural 
technology which so improves the lot of 
the farmer and the effidency of l^y; ag- 
ricultural process that it displace the 
marginal worker and forces him Ir to the 
city and leaves him there to sulk and to 
become discouraged and become dis- 
connected with the very sodely which 
that technology is supposed to 5 cr>*e?* 
*How Can a society be so Inhumvie as 
to bring new mcoical technology to a 

•rreii<fm<-lOerlgfwfe, Th^ Vnk<frit)f .if OkW 



country to decrease the birth rale, in- 
crease the life expectancy and decrease 
the death rate and allow those people to 
slar\'e?* The young person continues to 
ask, "WTiat is the value of a system that 
does not consider slow starvation as 
deadly and as devastating as a sudden 
death?* "What,” says the young fellow, 
"is the problem with a society that does 
not seem to be able to control the nature 
of the weaponry that technology brings 
— whether that weaponry Is nuclear 
bombs or chemical and biological war- 
fare?* That s one side of the question. 

And the other side of the question is 
that the devices of technology, the engi- 
neering achievements of man over thou- 
sands of years, thousands of centuries in 
his evolution, have made it possible for 
man to survive at all. It seems almost 
abundantly clear that uilhout weapons, 
man coula not have sunived and devel- 
oped in an c\*olutionary way the deep 
culture and the nature of the society. 
The wcajK)n became a tool, and the tool 
made it possible to remove much of the 
burden, at least of the affluent society, 
from man's back. It extended his arms; it 
plowed his fields; it helped in the treat- 
ment of disease. Technology has been 
the means by which it has bewme possi- 
ble for man to use bis creative talents to 
do something more than be a slave to 
the driving necessity of having food and 
shelter and clothing and protection. 

Technology has two sides. It has on 
the one side the immediate and direct 
benefits that it brings in the treatment, 
the caring, the feeding, the housing, and 
the transportation of people. And it has 
another side, and that is its potential 

a, OlJchoma 
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harmful effects on the society In general. 
This dichotomy, this difference in view- 
point, causes the deepest possible diW- 
sion in those people who are of good 
heart as to wnat is the proper use of 
technology for the benefit of society, 
and who is to decide? It strikes me 
strange, that in engineering, in medi- 
cine, and science, the pcoj^c who are 
concerned >Wth the application of tech- 
nology to the useful pursuits of man, are 
not at the same time deeply concerned 
uith the ethical, moral, political and le- 
gal questions which that very technolo- 
gy produces in its introduction. Let me 
ask you a few questions: Who shall de- 
cide when and under what conditions 
artificial devices will be used to extend 
man’s life? Shall we leave it to doctors, 
as presently trained, doctors wiro are 
highly capable and highly skilled techni- 
cal people, but who are largely not 
knowledgeable of the law, of the socie- 
ty, of religion, faith, values and ethics? 
Having had to spend so much time be- 
coming the doctor that he is, is It possi- 
ble for him to decide the deepest kind of 
value Judgments alone, such as who 
shall benefit from the technology which 
is just around the comer? Is it proper, 
for example, for a decision to be made 
as to which of two patients gets an arti- 
ficial kidney because one ot them hap- 
pens to have more life insurance than 
the other and therefore his family can 
1)C better cared for? F wonder. 

The technology which man has devel- 
oped has made it possible for the social 
change in every modem society. Tech- 
nology and its development led to the 
industrial revolution. Technology and iU 
development led to the development of 
the modern city. Technology and its de- 
velopment maae it possible for only 6i 
of Ine people in the United States to 
feed all the rest of us. Technology made 
it possible to handle information In such 
large quantities that no man can possi- 



bly compete with it as far as the storage, 
retrieval and codification of that infor- 
mation. Technology will, in fact, in the 
future make it possible for man to 
change the genetic character of men 
themselves. The technology that created 
the drugs so wonderful in the treatment 
of disease, is yet the very beginning of a 
technology which vv’ill so infuse educa- 
tion and medicine that we can speak of 
the present lime, not an industrial 
revolution, but as an information revolu- 
tion. We are at the stage in the society 
where the extension of man's arm by 
machines and tools is now nearly com- 
plete, but the extension of man’s minJ 
and perception in the seeing, evaluating, 
handling, storing, and projecting infor- 
mation has only begun. And if you think 
today’s sophisticated computers which 
arc able to program thcmsclv*cs are so- 
phisticated, if you think that information 
censoring devices that can be used re- 
motely arc sophisticated, I will remind 
you trial they arc yet but the very sim- 
ple and nascent beginning of what will 
DC, for we have now^ discovcrctl, like the 
discovery of steam power and the me- 
chanics of conv'crling heat to energy, 
that it is possible to manipulate, control 
and calculate the handling of informa- 
tion. And we have discovered simulta- 
neously that it is possible to do that on a 
vTcy small scale, on a scale that docs not 
yet reach the scale of the ncviral net- 
works of the brain, but on a scale which 
in a ^ccadc may be so. \S'c will be able 
as well to produce the kind of connec- 
tions in a computer, that will make it 
possible for one to say truly that a ma- 
chine thinks, rather than just solv'cs 
problems. 

All of these things raise the deepest 
kinds of questions — questions, first, as 
to how these techniques arc to be used 
and to what end, and how they shall af- 
fect the very people who use them, and 
what resislcnoe will the practitioners of 
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the old arts put up, for it is also true 
that technology, by being the mother of 
social change in many regards is also 
resisted deeply and often effectively by 
the very people who have to put the 
technology to use. I was reminded one 
diy of a conversation with President 
John Kennedy and one of his economics 
adWsors. We were trying to persuade 
him and the President that everything 
that w'as being done except the final 
evaluation of the state of tne economy 
by the economists could be done better 
by computer, and that we were not us 
ing computers to model the economy, 
but rather we were using some sort of 
subjective ad hoc judgment. Ill never 
forget the day we tried to persuade 
them that technology would make Nviscr 
judgment possible by economists if they 
would but use it to model the economy. 
The Economist said to me, ^Hcrb, but 
then w'hal w'ould I do?^ And you know, 
that’s sort of an odd reaction — a reac- 
tion, however, that is completely under- 
standable, for all of us depend for our 
security on the things we’ve learned 
when young, and we depend for our 
jobs Oil the tochniquci and tools of a 
decade or tw o ago. 

Technology, specifically in medicine, 
will lead clearly to all sorts of artificial 
dcrices and implant:*. And here the 
uesUon comes, just as in the case of 
rugs, under which conditions shall they 
be used, what tests shall bo required 
and what judgment shall be made? 
These questions will be applied, even 
more specifically to computers, and I 
don’t think the computer is nearly as im- 
portant as the whole process of handling 
Information. It is the whole s)*slem. It is 



the conversion, the calculating, the scor- 
ing, the retrieving, the disp!a)ing, and 
the orderly logical sequence of the proc- 
ess. All too often, computers are used 
simply to replace manual operations. 
The computer prONidcs, however, a 
technique and a possibility of doing 
things with information that man canT 
do himsdf at all. And that’s why it is so 
impo. cant for us to consider how com- 
puters and information systems will in 
fact affect diagnostic and clinical aid, 
work on organizational or administrative 
problems, and most importantly, aid 
educational tools. But they will not work 
if they are a mere substitute for what 
humans arc now doing, and they will not 
work unless the computer and the Infor- 
mation system is deeply tied to the N*ery 
people w^ho have to use the information 
once it Is produced. Information systems 
In medicine will make it possible to aid 
large numbers of people with relatively 
fewer professionally trained medical 

3 1c. That doesn’t mean we won’t 
more trained medical people, but 
the relative number should oc smaller. 

One of tho things that is so important 
in this country is that we have commit- 
ted ourselves to two deep responsibili- 
ties about people. One is, that they shall 
have the opporlunilv to have all the ed- 
ucation that their ability and motivation 
allowr, them, Independently of their 
wealth, sex, and color. This ideal is very 
costly. The second ideal is that no man 
should be v.ithoul adequate medical 
services because of his position, his race, 
his color. If we arc to do these two 
things at a price wo can pay, our only 
hope is to use Icchnolog)' in its most 
beneficial way. 
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OBJECTIVES OF CONFERENCE 

JOSEPH M. WHITE, M.D.* 



Mr. Chairman, Ladies, and Gentlemen: 

r wish to add my welcome to that of Dr. Dennis and Dr. Hollomon. I sin- 
cerely hope that you will enjoy your visit with us and that you will find the 
conference both interesting and stimulating. Our increasing capabilities to dis- 
cover new uses for the modem computer plus the enormous efforts now being 
expended in almost all our medical schools in modification and revision of the 
medical curriculum make this conference a timely one. 

The use of computers as an educational tool is both direct and Indirect. 
Indirect applications include the support of the administrative and research 
activities of a Medical School and, more recently, the processing of patient 
care information. Direct applications to medical education are now beginning to 
appear. These include information retrieval and the use of the computer as an 
instructional tool. It is this latter role that we will be concerned with in detail 
during this conference. However, we must keep the other educational applica- 
tions in mind and fully explore the potential for them. 

In any instructional system, three general problem areas must be considered. 
These have been described as the technical, semantic and effectiveness prob- 
lems. The technical problem is concerned with the hardware and equipment 
capabilities required to implement the system. The semantic problem has to do 
with the relationship between the meaning intended by the instructor and the 
interpretation made by the student. That Is, design of a system which will ac- 
curately convey the Instructor's thoughts to the student. Finally, the effectiveness 
of an Instructional swtem Is concemw with those changes induced In the student 
by the Instructional experience. Each of these should ^ carefully explored be- 
fore introduction of any new system. 

Before discussing the specific objectives of this conference, it would be well 
to mention some of the thoughts of others regarding the role of computers in 
an educational setting. Three major roles have been defined: 

1. to sene as a resource to the student for study; 

2. to assist the instructor in his own study and preparation and in prov^ion 
of timely summaries of student progress individually and as a group; and 

3. to provide assistance to the learning researcher. Some experience has been 
gamed In each of these areas, especially in the undergraduate college 
curricula. 

The following general questions must be explored if the objectives of this 
conference are to be realtied- 

1. What is now being done with computers in Medical Education? 

2. What are the realistic potentials of computen in hfcdlcal Education? 

3. What ty^pe of work must be done to realize that potential and how should 
this proceed? 

•Oran, hlcdkcl TcctJty, VnkettUy cf Oktohom» Center 



These general objectives invoke a number of more ipecific cx>nsiderationi 
including: 

What arc the technical problems that must be overcome? What constitutes 
an effective system? 

Perhaps more importantly, what requirements are placed on the faculty and 
students to enable them to make effective use of the computer based system? 
How can training programs be developed to permit them to meet those require- 
ments? 

Will students and physicians accqpt the direct use of computers as a part of 
their educational process? More specifically, will they utilize tne potential of the 
computer by silling at a terminal and interacting with an instructional program? 
How do we get them to take advantage of it? 

How can the total effectiveness of this means of instruction be evaluated 
and, obviously, it musll 

How should such a computer based inslructicnal system fit into a modem 
curriculum? Should it serve primarily as a learning resource for the student or 
can it in fact be a basic part of his primary instruction? Should such a system be 
independent or should it dc a part or a larger information system? 

Although it is not possible at this time to provide the specific answ ers to all 
of these questions, we are hopeful that this conference can generate ideas and 
directions to follow if we are lo find the ans^vcrs. 

The conference has been structured so that ?ach of you will be able to par- 
ticipate actively in the discussion and as ?ducalors vour ideas are needed. We 
feel that the more or less Informal nature of this conference will permit an envi- 
ronment leading to full opportunity for you lo express your own opinions and 
ideas. If this conference is to bo a success it will mean that all sides of the Issue 
have been eiplaed. W^e arc hopeful that this will be done. 

Thank you very much. 
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STATEMENT OF THE SURGEON GENERAL* 



>V1LLIAM H. STEWART, Nf.D. 



Medical education is challenged today as nev^er before in its history. 
The needs and aspirations for medical care expressed by the American 
people demand rapid quantitative growth. Our professional commit- 
ment requires that this growth must not be accomplished at the ex- 
pense of quality. The explosive development of m^cal science and 
technology, by raising the potential for excellence ever higher, makes 
the production of such excellence increasingly complex and difficult 

For these reasons it is \itally important that medical education draw 
upon the best in educational lecnnology. We simply cannot meet the 
new challenge with the old familiaj' methods. Yet, at the same time, it 
is equally imperative that we be selective in adopting and adapting 
new metnodology. We cannot afford to accept new methods just be- 
cause they are new^ or abandon old ones bwause they are old. We 
need to measure such means aga-nsl our fundamental purposes and 
choose accordingly. 

Therefore, this conference is not merely timely; it is of the highest 
urgency. In wishing you a most productive and successful meeting, I 
am wishing more than that: that this conference may represent a 
significant stride toward more productive and successful medical 
caucation In the years ahead. 



^Trcnmtilfd to Conference on the V»e of Computers in Sfedicct Kducctkm Frank W. McKee, 
M.D., oi penary tcnk>n. 

O 16 




.15 



EXCELLENCE - BUY OR BEGET? 

FRANK W. McKEE, M.D.* 



It is alwfiys a pleasure and privilege 
to parlicipilc meetings deoiealed to 
the study and improvement of medical 
education, and the exploration of belter 
and more efficient ways of coping with 
thiS fundamental responsibility* I am 
particularly Jionored to be invited to 
speak before you bcg.'n your work in 
pursuit of the ‘'Objectives of the Confer- 
ence** that lia\'tj just been laid before 
you, because this offers me the oppoftii- 
nily to share with you some broader per- 
spectives of met! i cal education into 
which these immediate objectives and 
your activities in support of them may 
fit. I should point out the rather obvious 
fact that my position on the program 
should convey to you that I am not in 
any sense an expert fn the use of com- 
puters as an educational tool, or will 1 
present any new experimental or other 
data about the subject of the confer- 
ence; otherwise I would 1)C included 
with the quality cast you have assem- 
bled for that pvirpose. Therefore, 1 hap- 
pily accept my role as a layer of founda- 
tions, and will watch with interest the 
emerging structure of new contributions 
to the medical education process which 
)*oii erect from man-made tools and sys- 
tems. 

As rnanv of you know’, lam privileged 
to head the Division of Physician Man- 
pow’cr in the Bureau of Health Xfan- 
power. This Division is the focal point 
tor medical education in the IVblic 



Health Service. Within the responsibili- 
ties of our assignment arc those related 
to the major support of educational pro- 
grams, both in medical school curricula 
and in continuing education activities, in 
the construction of new schools for all 
the health professions, additions and ren- 
ovations to existing schools, and in sup- 
port of hospital and patient care facili- 
ties which arc intimately concerned with 
the educational process, The newly pro- 
grammed Basic and Special Improve- 
ment Grants, which arc the Federal gov- 
crnmcnl*s first clearly defined funds for 
support of educational operations, are 
within our review mechanisms. Through 
research grants and contracts, support 1$ 
offered for a variety of investigations of 
new approaches to educational format, 
with the pious hope of a dividend of ac- 
complishing some progress in the educa- 
tion of both medical students and prac- 
ticing physicians while the experiments 
go on. All of these efforts arc compo- 
nents of the overall mission of the Bu- 
reau of Heallh Manpower which is di- 
rected towards a quantitative and quali- 
lalivc improvement In all the health 
professions in the provision of medical 
care for all the American people. In a 
nutshell, vve arc interested In increasing 
the number of qualified individuals who 
will responsibly take care of the sick, 
and act effectively in disease prevention 
and control, notwithstanding the other 
important considerations for develop- 
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mcnt of new knowledge and ihe on- 
going study of disease processes. 

Having outlined, then, the specific ob- 
jective of increased numbers of Qualified 
manpowefi and having described the in- 
terests of my particular Division in the 
physician portion of this goal, and the 
various ways in which the production of 
physidan manpower is supported, it Is 
appropriate to offer some observations 
about the educative process itself and its 
dominant and pervasive role in the 
achievement of these national objec- 
tives. Tlic following comments are not 
based on second hand or hearsay infor- 
mation, for, in dealing with all of the 
colleges of medidne and osteopathy, 
and having the privilege of assisting our 
national legislators and others in rc- 
pl)ing to a wide spectrum of Inquiry 
and criticism of present medical care 
systems in the United States, one devel- 
ops a reasonably accurate, continuing, 
an f > vthcr acute familiariiy with what a 
varlci^ >*'f our citizens think is vvTong or 
lacking. In our abllily and agility to re- 
spond to such Inquiries, a number of 
problems are further identified and pos- 
sible solutions arc generated. 

The concerns of the public, as pre- 
sented in correspondence, and In other 
inquiry, fall into two categories. The 
first is that there are not enough physi- 
cians doing the right things in the right 
places. Rural areas, and Inner dtics 
seem to be ibc most easily identified des- 
erts of beaUh care, lut other geograph- 
ic areas and even our ulopia of modem 
living, the suburb, has its problems. The 
fourth branch of the Division of Ph)*$l- 
cian Manjx)wcr, to which I have not 
hcrclofoTC alluded, Ph)*sician Utilization 
and Supply, Is concerned with solutions 
to the problems of ph)*sician distribu- 
tion, through encouragement of studies 
of more effective practice patterns, 
invTstigations of why ms^r gaps in 
health personnel exist, and the means 



that might be best used to correct these 
unfortunate, and occasionally desperate, 
situations. 

The second major concern of the pub- 
lic relates to the inability of large num- 
bers of apparently eminently qualified 
young men and women to gain admis- 
sion to medical school. IjCtters on this 
subject arc directed to a number of gov- 
ernment official; with pleas for interces- 
sion on behalf cf this or that individual, 
a practice which is consistently regarded 
as meddling in the affairs of a school or 
schools. But, be that as It may, when 
there Is public clamor, there will be ac- 
tion, and it is our hope that the expan- 
sion of established schools and the con- 
struction of new scliools will provide 
increased educational opportunity for 
every qualified applicant in the United 
States. 

These two citations are foci of public 
attention and represent a surfacing of 
deeper and more complicated concerns 
about medical education which are in 
the purview of all of us vvho arc In- 
volved both in providing it and support- 
ing It. These latter proolcms arc many, 
with a large overlay of human nature in 
them, which often, surprisingly, is more 
serious than financial problems. Of late, 
a number of cliches to Identify clemenls 
of these problems have crept into the 
language of medical educators, and 
while some of these arc tiresome and 
irksome and subject to interpretation by 
the one using the term or phrase, they 
do bespeak ^th complacency and, par- 
adoxically, a certain insecurity and 
"whistling in the dark to keep up cour- 
age" sort of attitude. Since I hardly have 
time today to run through a glossary’ of 
terms used bv medical and other cduca- 
tori to descrioc the ideals, the goals, the 
problems, and the undcrl)1ng finandal 
TKeds, )*ea, even demands, for their 
achievement or solution, I will mention 
only one ~ "EXCELLENCE." 



Excellence Is aboul as complete a 
word to define educational aims as one 
could wish, and has as synon)ms — su- 
periority, preeminence, worth, value, 
goodness, purity, and greatness. Its es- 
sential meaning Is merit and \irtue, and 
could well join with motherhood, peace, 
the American flag, and the reduction of 
taxes as a standard under which we 
could all join. Excellence apix'ars with 
an element of glibness in mcoical school 
publications, particularly those directed 
tow'ards alumni and granting agencies, 
and 1 tpjolc one such ; 

**Jn tne face of pressures for change, it 
is especially important that we keep our 
primary goals and responsibilities clear- 
ly before us. 

'The first of these is excellence in ed- 
ucation. This arises from the capabilities 
of the staff, the quality of the student 
body, and the continuous upgrading of 
our educational program. It arises, tc*o, 
from the commitment of the staff to re- 
cognizing the indiudual qualities of 
each student and building upon them. 
Although we will not invariamy be suc- 
cessful, we strive for an cnWronmcnl In 
which understanding and cnjo)nncnt of 
education are shaiM by students and 
staff alike. As pressures rise for the edu- 
cation of more ph)*sicians, we must ex- 
amine our educational goals, explore In- 
novations, and establisn our programs 
on the basis of the best cWdcnco avail- 
able,* 

This Is what f would call a complete 
statement, and while an expression of 
but one school, It could well serxe all, 
and be a preamble to whatever might 
follow’ as tlefinltions of accomplishment 
or requests from any and every source 
for sup|>ort for faculty, students, and fa- 
cilities, It is hard to fault as an expres- 
sion of goals and Implies all of the defi- 
nitions and s>’i>on)*ms of excellence In Its 
staled quest. But while it can escape 
criticism as an ideal, it nnosl certainly, to 



those w'ho know about processes of im- 
plementation, lays itself bare to searching 
question, and tne *feet of clay* on the 
quicksand of its major premise. In pur- 
suit of excellence, medical education has 
assumed a flavor of Madison Avenue 
and the supermarket, with “capabilities 
of the faculty” prominently mentioned 
as a pillar in generating “excellence in 
education.* 

Perhaps these “capabilities* should bo 
scrutinized more closely to determine 
what they arc, what they should be, and 
whether they arc actually directed at 
the purposes of medical education and 
its ultimate goal of helping solve the 
problems of a real w*orld of people, dis- 
ease, and untimely death. 

Some )’cars ago, before the enormous 
investment of the Federal Government 
in support of medical research, medical 
schools followed Integrated programs 
embodying leaching, research, and pa- 
tient care with minimum categorization, 
because of the recognition of the impor- 
tance of all these elements in physician 
education. In recent times, however, this 
categorization has been carefully identi- 
fied, the components have been fcgisla- 
tisTly isolatco, and the integrity of the 
school as a common center for medical 
education has been greatly changed. 
Tills situation has led, in turn, to con- 
sidcrablo change In the faailly, those 
who constitute it, and what the Ideas of 
both the school and the faculty member 
arc aboul correlated responriniffty. Hc- 
cruilmcnt of faculty Is now’ a highly 
compclilivc exercise for trained indixid- 
uals In short supply and Is characicrized 
by high Interest and investigation of al- 
most oer)lhing about ibe candidate 
that has to do with prestige, funds, and 
personal contribution, In varying mean- 
ings of the term. Strange and unwar- 
ranted assumptions aboul teaching qual- 
ifications, character as a man, and as a 
model for students, are often made, and 
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may be covered by a statement such as, 
“He's a nice guy, and the students seem 
to like him.’* The candidate himself is 
usually more captivated by the personal 
advantage of the situation, and concerns 
himself with his space, his arena of con- 
trol, be it medical specially or research, 
his teaching “load,’ not “opportunity,’* 
and what colleagues he will find worthy 
to help continue the rise of his star. 
From this sort of actirity, then, comes 
one aspect of faculty capability, which 
is hardly conducive to a program of 
medical education directing young men 
and women to preparation for assuming 
major rcsix>nsibililics in the care of sick 
people. 

A second factor In faculty capability 
is the actual involvement of patient care. 
At one time, all medical schools had 
numbers of competent full-time serious 
clinicians, who not only taught medi- 
cine, but personally practiced it. They 
were backed up bv a substantial part- 
time or volunteer faculty who were ac- 
tually engaged in private medical prac- 
tice, and brought to the students and 
house officers a true flavor of day-to- 
day practice to complement other lessons 
of greater breadth or depth. With the 
growing ability, and Incidentally, real 
need to attract funds through the mech- 
anism of research and Its considerable 
support, more and more full-li’me faculty 
were addctl to the medical schools. Con- 
comitantly, burdens of increasingly de- 
manding practice greatly diminished the 
lime available to the private practitioner 
to participate in rcgulai teaching sched- 
ules. But whatever the factors responsi- 
ble, this change in personnel has done a 
rave disservice to many students who 
ave a conscientious desire to become 
accomplished practitioners outside a 
university or medical school setting, and 
has effected a considerable diminution 
in “capability of the lacuUy* in its 
broader fcnsc. Some expanded use of af- 



filiated hospitals has compensated to a 
degree for this change, and perhaps ex- 
plains the popularity of these programs, 
a popularity that often mystifies the 
full-time faculty member in his universi- 
ty hospital office. 

A third factor in determining “capa- 
bility of the faculty” is whether there is 
any general understanding of the rea- 
sons for the existence of a particular 
medical school, and what its major inter- 
nal role might be, as well as its responsi- 
bility in the local, stale, or national com- 
plex. Such understanding should involve 
the President and the Board of Trustees 
as well as the Dean and the faculty. In 
some instances, where a single school 
exists in a given stale, this responsibility 
should be quite clear, but I nave been 
amazed in a recent visit to one such 
stale to find the wide differences of 
opinion of the slate legislators and the 
medical school administration as to the 
fundamental purposes of the medical 
school. However, if the true goals of Iho 
school are defined, and generally agreed 
upon, and if there Is tne Intention to 
produce qualified graduates, who will 
contribute to the real world of dealing 
with illness and its prevention and con- 
trol, and who will be educated to devel- 
op high standards of rcs^xmsibilitv and 
human concern, then the “capability of 
the faculty” (s more easily attested, and 
the achievement of excellence is a real 
possibility. How strange it seems, in this 
day of heart transplants, Nobel and oth- 
er prestige prizes, and enormous indi- 
vidual project grants, that excellence 
can be pursued as a goal within itself. It 
actually can be more closely approxi- 
mated by accepting a clear and obvious 
responsibility, meeting it, and finding 
satisfaction in a job well done. 

Excellence, true excellence, was once 
developed ar>d nurtured as a home- 
grown product and expressed itself in a 
dedication and loyalty to the institution 
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and understanding of its purposes which 
was truly representative of "capability 
of the faculty.” The teamwork was the 
result of years of associations with es- 
tablished trust and mutual interest. The 
final product — graduated physicians — 
reflected a stability, responsibility, and 
cont'cm which was true excellence. Tfic 
World Series will alw'avs be a better dis- 
play of baseball than Inc All-Star games, 
and the school that begets its own excel- 
lence will always have more to offer its 
students and the people for w'hom it is 
responsible, than purchased ''excel- 
lence.” 

From this inherent capability, then, 
comes the greater wisdom for designing 
a d)'namic, competent, and inspiring ed- 
ucation program, which might really be 
accepted by the faculty rcs|X>nsible for 
it. The "quality of the student body" 
cannot fail to be enhanced, with per- 
haps some resolution of problems of ad- 
mission and the alarming attrition of ad- 
mitted students, and some true student 
enthusiasm for "the continuous (ly) up- 
graded educational program." Concern 
For direct patient care, coupled with 
good educational tcchniaucs, will amel- 
iorate public anxiety aooul physician 
supply and availability. Such concern 
will also have a salutary effect on the 
wasteful and distressing problem of in- 
creasing student attrition. 

A profession as essentially human as 
medicine must have a firm base in the 
character of its primary scat of produc- 
tion — ibe medical school, for it is fiom 
this foais that the true quality of profes- 
sional excellence emanates. 

On behalf of ibis ideal, it is the policy 
of this Bureau and Division to support 
institutions and strengthen Incm 
through their chosen administrative 



channels, so that the determination of 
the essential educational need is neither 
dissipated nor eroded by less important 
considerations. It is equally important 
for the Association of American Medical 
Colleges, acting through its individual 
constituents, to review the whole spcc- 
Inim of faculty acquisition and to en- 
courage principles and procedures more 
consonant with the needs and purposes 
of the system. 

The role of this conference is to dis- 
cuss among knowicelgcablc people an 
exciting adjunct to the processes of med- 
ical education, which can offer real as- 
sistance in solving problems of increas- 
ing information and demands for accel- 
erated production. I lewk forward to 
some interesting revelations and accom- 
plishments, but the keystone of this key- 
note address is that the success or fail- 
ure of our system of medical education, 
in its multitude of individually tailored 
and variously directed programs, de- 
pends on the cbaraclf T and the quality 
of its leaders and proponents. The re- 
finement of proccthiro, the Innovation of 
method, the desire for excellence as ex- 
pressed through graduates dedicated to 
medical care in all its facets, will always 
rely for their success tjpon the human 
Ix'ings who choose to accept rcsj>onsibil- 
ily for the achievement of the goals of 
the profession, not the particular whim 
of a certain school or individual. The 
goals of the profession were dcfinc<l 
long ago hy Hippocrates, Jesus of Na?a- 
rcth, Maimonidcs, and have hcen ear- 
nestly supported through the ages hy 
considerable mi mixers of men of dedica- 
tion and principle. Perhaps we should 
all devote more attention to this funda- 
mental matter, and I would urge your 
personal consideration. 
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EVALUATION OF THE USE OF COMPUTERS IN 
MEDICAL EDUCATION 

STEPHEN ABRAHAAfSON, PhD.* 



The use of emerging computer tech- 
nology in education is, of course, quite 
recent. And, as is the case of the appli- 
cation of educational innovation to med- 
ical education, the introduction of com- 
puters as an aid to medical education 
conies significantly later. Thus, when 
one asks questions concerning CLVi/tw- 
tion of the use of computers, answers 
cannot really be found In data at this 
time. At least, data in substantial and/or 
significant amounts do not exist. Thus, 
this paper cannot assume the familiar 
form of a review of research or of a syn- 
thesis of published (or unpublished) re- 
ports. Rather, at this stage of develop- 
ment of the application of computer 
technology to medical education, one 
can only look forward to the design of 
studies and collection of data. 

Consecjuenlly, exercising familiar li- 
cense, the author Is changing the topic. 
The modification truly reflects the 
chronological sequence of events In edu- 
cational Innovation, incidentally, and at- 
tempts to warn — through some discus- 
sion — of the days ahead. Wliat factors 
must be considered as evaluation Is 
planned and experiments arc conducted? 
What data are needed to help us make 
informed judgments? WTiat kind of re- 
search design will pro\-lde an adequate 
test of the use of computers in medical 
education? 

Despite the fact that millions of words 
of glittering prose (arxl possibly soith; of 



poetry) have been written on the topic 
of evaluation, some review is essential at 
the outset so that later discussion will 
not become snagged in the ibistlcpatch 
of conflicting d^nitions. All loo often, 
mutual understanding in ibis area can- 
not be achieved because of differences 
in just plain definitions. 

Avoiding the use of technical jargon 
and an extended presentation of a for- 
mal definition (evaluation Is a contin- 
uous process, based upon criteria coop- 
eratively developed, and concerned with 
the status of and changes In students' 
l>chavior) one must still acknow^lcdgc 
the steps in evaluation. No matter how 
many times it has been said before, no 
matter bow many different authors have 
written these things, evaluation still con- 
sists of these component parts. 

(1) Euilvation still (he 

definition of ob/ecricei. Tms statement 
has been made so often and by so many 
people that listeners (and readers) 
weary of hearing (and reading) this 
dictum. But It is nonetheless truci It Is 
nonetheless important; it is nonetheless 

— unfortunately — disregarded. It often 
seems that because wo have "talked 
about objectives" for so long that wc 
have somehow — through some catharsis 

- rid ourselves of the need to define ob- 
jectives. Such is far from truth. With re- 
gard lo any educational program or ex- 
ercise the questions must be raised (and 
aruwered); Wliat should the students 
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know at the end of this educational ex- 
perience? \Miat should thev know how 
to do? IIow else should they bo 
changed? 

(2) Evcluation still demands criteria 
by which to fudge students* achieve- 
ments of those objccUves, ^Vhile these 
arc sometimes embodied In a good state- 
ment of objectives, mo\c frequently they 
arc omitti'd. Ifow much new in/orma- 
tion should students have aexjuired? 
How proficient should they have be- 
come in the '■;(ocution of specific skills? 
ibese questions should he answered, In- 
cidentally, before the attempt to meas- 
ure achievement and not afterward by 
scrutinizing scores of other records and 
assigning some “logic d” (statistical or 
otherwise) ciitolf point or points. 

(3) Ecaluation still rcqtdrcs t7icasure- 
ment of the achievevu nt of each of the 
obfcciivcs. And educational measure- 
ment is lx‘St accompMshed unt-dimen- 
sionally. That Is, acqu sition of Informa- 
tion is most approprI;dely and icUably 
me as ur cel by appli ’a tion of some 
achieve ment -lest Iccinique — jx-neil- 
and-paper, perhaps. 11 c measurement of 
achievement of skills necessitates some 
observation of the stu lents in perform- 
ance of those skills — c»lhcr dirccily or 
through the aid of electronics in the 
form of \ideo-tapo. But nicasurenient, in 
whatever form, providi s the information 
concerning student po^gress needed for 
the fourth step. 

(4) Evaluoticn sUV includes imple- 
mentation in the form of action involv- 
ing the student, the pr ogram, the objec- 
tives — or even the pro'c sslon. One must 
study the results of measurement in the 
light of stated objectiv e, and according 
to the criteria make dc< isions — and take 
action. Hire is the judginenlal phase of 
evaluation. B\d it snould not stop short 
of the accompan)nng art 'on to roise the 
program, resiew the (bjcctivcs, certify 
the students, or initial • new studies of 



students, program, or profession. 

Concerns in evaluation of the use of 
computers in medical education are per- 
haps best expressed in these four ques- 
tions. 

1. Does the use of computers facilitate 

learning? 

2. If so, does it do so at the expense of 

other objectives? 

3. WTiat is the cost of the use of com- 

puters to facilitate learning? 

4. Docs the use of computers result in 

less faculty and/or sl\ident input of 

time and/or effort? 

Essentially three areas of study are in- 
volved: (1) effectiveness, (2) cost, and 
(3) efficiency. The first two questions 
above deal with effectiveness, while the 
third and fourth refer to cost and effi- 
ciency, respectively. All of those become 
major concerns in any educational ven- 
ture — but particularly in the situation 
involving the introduction of innovation. 
There is no quarrel with the contention 
that conventional and standard and tra- 
ditional mctliods in education should be 
studied critically. For all too long ctlu- 
cation has enjoyed the luxury of as- 
sumption of achievement rattier than 
having to test its processes critically. It 
has iK'cn said that physici.ans have such 
an enviaVde record of achievement In 
application of ihdr profcssioi:al |Kr- 
formance perhaps Ikc.uisc half their pa- 
tients would get l)clter anNway. It is un- 
doubtedly true also, in tnal vein, that 
teachers (and traditional education) 
can lx)ast of havnng achieved instruc- 
\ional goals because students would 
learn anjavayll Thus, all education 
rhofi/d be subject to careful study. But 
tho real burden Is vqx)n the innovator, as 
we all know. The fntrcKluction of change 
in teaching methods must he accompa- 
nied by careful study. Especially as we 
operate in areas involving increased 
costs ^or potentially increased costs — |s 
it incumbent upon us to document our 
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success, or lack lliercof. There is one 
comforting thought, incidentally. The 
more \vc attempt to study impact of ed- 
ucational innovation, the more we man- 
age to learn about conventional educa- 
tional processes. The more we apply 
Ourselves to tasks of educational mea- 
surement with regard to studying out- 
comes of newer methods, the more re- 
fined our measurement techniques be- 
come ~ and these can then be applied 
in an\j educational milieu. The message 
really is to avoid any sense of paranoia, 
to avoid a posture of ‘'why me?" when 
asked to justify — In educational out- 
come terms — the use of computers in 
medical education. 

Effcctkcncss — Docs the use of com- 
puters facilitate learning? If so, docs it 
do so at the expense of achievement of 
other educational objectives? The study 
of effectiveness cannot even Ijcgin, let 
alone proceed, without a clear statement 
of objectives. One cannot determine 
whether learning has been facilitated 
without tlic answer to a prior question: 
learning of what? At the same time, one 
must keep in mind that there are usually 
many objectives being sought — al- 
though perhaps not all in that sjKcific 
application of instructional media. Kev- 
crthclcss, they must be considered. For 
instance, if a tcxhnifjuc is employed to 
facilitate the actjuisition of information, 
it is fmjx>rtant to know, of course, 
whether it did so. It is equally imjx>r- 
tanl, however, to discover whether the 
use of that Icchnicjuc ofjo (accidentally, 
inadvcTtcnlly, incidentally) negatively 
influenced student altitudes toward 
learning, toward continued study, or 
even toward particular asjKC-ts of pa- 
tient care, ft is [X)ssiblc for one learning 
experience to pronrrotc achievement of 
on? set of olqcciivcs while significantly 
detening achievement of another. Sluc(- 
Ics of effectiveness should try to include 
some measurement of achievement of 



other objectives, hopefully representa- 
tive of all of the educational objectives 
of the total educational effort. In a 
sense, rival h)pothcses must be stated 
and tested. 

In studying effectiveness, investiga- 
tors must also avoid the error of testing 
for educational gain only at the comple- 
tion of the instructional process. I low 
long do students retain the new informa- 
tion? Over how long a period of time 
can students still perform the skills they 
learned? Data collected in other studies 
suggest rather strongly that retention 
may be related to ccriain other varia- 
bles: utility, motivation, experience. 

Studies of retention of information and 
skills gained through the use of comput- 
ers in medical education, thus, will help 
in the understanding of this aspect of 
education. 

Cost — It seems a shame to have to 
spend time on the question of cost. 
Here, again, is a question not really 
raised with regard to many other mcxles 
of instmetion. Unfortunately, the bur- 
den of proof is on the innovator. And 
tlic initial cost of equipment plus the ex- 
pense of operation and maintenance are 
not insignificant. In addition, there is 
the area of preparation of “software” — 
the subject matter preparation, the pro- 
grams themselves. The cost in dollars 
and cents must be studied and included 
in evaluation. 

But that’s not all the cost that must be 
studied. In addition, the input of faculty 
time and effort in monitoring, in evalu- 
ating student progress, in Just learning 
how to prepare programs and use the 
computer h.irdwarc must l>e considered. 
Finally, revision and modification costs 
must also l>o calculated. Such costs in 
convcnlion.il education arc minimal and 
frankly not considered. Wlicre the tech- 
nology demands effects beyond the T>or- 
mal range, a systematic examination is 
in order. 
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Efficiency — Docs the use of comput- 
ers affect faculty time and effert neces- 
sary to achieve the educational goals? 
By the same token, what Is the effect on 
student input of time and effort? Essen- 
tially, a ratio of “outpuf* to “input’' is in- 
volved and should be studied. Tlic study 
of output must include such factors as 
number of students educated, how much 
they learned, how long they retain what 
they learmd, how wcW they apply what 
they learned, how much they* arc stimu- 
lated to study more, and tlie like. The 
study of input Includes such factors as 
how^ much lime the instructors invested, 
how much lime the students devoted to 
the learning process, how many faculty 
meml)crs were Involved, how long It 
took the students to reacli required per- 
formance levels, and the like. It Is possi- 
ble that an “effidency ratio** or an ‘^effi- 
ciency index** must be developed and 
applied. Without It, statistical design 
must be employed to allow this area to 
l>e explored intelligently, for it is the 
area In which many claims have been 
made — and will continue to bo made. 

Another Interesting area of concern 
must Ik? exposed and explored Ix-forc 
this disciission moves on to suggested! 
study design. This Is the area of concern 
for patient safely and comfort. One per- 
sistent educational problem has always 
Ixrn how to have Ix'ginning students 
learn skills, the performance of which 
poses a threat to the safety and/or com- 
fort of the human object, the patient. At 
the University of Southern California, 
this concern led to the development of a 
.^omputcr - controlled anthropometric 
manikin — a patient simulator, which 
has come to ho known as "Sim One.** 
Thj original piiri>osc was to provide a 
simulated patient on which neginning 
■**>cslhcsSology residents might Icam the 
skills of endotracheal Intubation prior to 
their having to perform the intubation 
of real, live patients. At least, the moth 



vation to begin this interesting w’Ork lay 
In this area of concern. The stated pur- 
pose of the project w^as to test the feasi- 
bility of simulating a living patient, ca- 
pable of responding in a lifelike manner 
to things done to it. In addition, effec- 
tiveness of its use in training anesthe- 
siology residents is being studied. 

Sim One is quite life-like in appear- 
ance, having a skin-colored, skin-tex- 
lurcd plastic covering in the form of a 
real human being. Sim One has the con- 
fig’jralion of a live person from the 
Wi.Ist up and Is In the posture of l)ing 
on an operating table, left arm exterided 
and ready for intravenous Injection, 
light arm fitted with blood pressure 
cuff, and chest w’all adorned w’ilh a 
stethoscope taped over the approximate 
location of the heart. Sim One breathes: 
has a heart beat, temporal and carotid 
pulse, blood pressure; opens and close.s 
his mouth; blinks his eyes; responds to 
four intravenously administered dnigs 
and two gases administered by mask- 
ard-bag or by lulv. The important thing 
to note here Is that Sim One has been 
used in the training of residents and ef- 
fectiveness tests reflect a great potential 
conlribiilion to hiirian safely and com- 
fort through having beginning students 
make their early errors and gain their 
proficiency and confidence on a com- 
pjtcr-eonlrollee! "leaching aid” rather 
than on a real patient. 

The lest of effectiveness involved five 
pairs of residents. These were assigned 
at random in such a way that five resi- 
dents were allowed to use Sim One and 
five were not. The measure of proficien- 
cy was a simple one. Hospital anesthesia 
records were cx^^mined by the anesthe- 
siology staff Hi>d a judgment was made 
a1)Oul each: \\*ould )‘Ou tnist thfs resf- 
dent alone In the operating room on the 
basis of performance like this? Needless 
to say, n.imes and dales were concealed 
from the rater. One criterion selected 
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was the achievement of four consecutive 
"'plus” ratings; that is, the rater u'oulri 
trust the resident in the operating room 
alone on the basis of what he reviewed 
in that chart, While the final analysis of 
data is not available at this moment, 
preliminary review indicates that it 
takes the non -Simula I or- trained residents 
tw'icc the number of trials in the operat- 
ing room (o reach that criterion level of 
performance when compared to the sim- 
\ilator-traincd residents. Such a differ- 
ence obviously represents a substantial 
difference in risk to pallenlsl! 

It should be noted, incidentally, that 
Sim One represents a remarkably imagi- 
native application of computer technolo- 
gy to problems of medical education. 
Ilcre is simulation at its most sophisti- 
cated; a realistic body with life-like re- 
sponses under computer logic. The dol- 
lar-cost of development has been high 
and the cost area in evaluation is indeed 
not on the positive side of the judgmen- 
tal ledger. Effectiveness and efficiency 
\sill be bomc cvl by data but the combi- 
nation of cost anu effectiveness might 
be found formidably negative in Impli- 
cation. However, the cost factor 

— patient safely — must also find Us way 
into the total cost-cffcctivcncss equa- 
tion. Thus, there are i^ow four areas for 
study; effectiveness, cost, efficiency, 
and himvin safety, 

How should evaluation l>c conducted? 
The best single, simple answer is a for- 
mal, carefully controlled, exjx'ri mental 
study. Not only wiW evaluative data be 
forthcoming, but comparisons with 
conventional educational methods will 
l)C possible. In this approach, one must 
attempt to control for all variables ex- 
cept one: the use of computer. Then 
with valid and reliable measurement of 
aebievement level taken before and after 
the conduct of the educational experi- 
ence, the investigators can document 
whether the use of computers did in fact 



‘facilitate learning.” And it must be 
pointed out that this kind of study Is 
most likely to catch the attention of and 
convince the skeptics — if the use of 
computers docs make a difference. Since 
the men who leach in medical schools 
arc themselves scientists, they constantly 
seek data in support of Ideas. Education 
is no exception for them in this regard. 
Furthermore, the .nreas of educational 
achievement studied should reflect the 
value structure of medical school faculty 
m cm hers. In other words, if achieve- 
ment in basic science areas is important, 
mcasvircmcnt of achievement in this 
area should be included — always re- 
membering, of course, that this is true 
only to the extent that such achievement 
is part of the objectives. 

Unfortunately, ibis approach is the 
most difficult to achieve in cvalvialion. 
Individual differences among students 
pose the first prohlcm; faculty differ- 
ences, the second. No two schools are 
really alike enough to allow for one to 
be usc<l as a "control” for another. Intro- 
ducing change of any kind also manages 
to change the total educational selling 
so that sjX'culalion can bo raised con- 
ctTning \\bethcr measured change in 
achievement is really only a reflection of 
the act of change itself: the so -called 
‘halo" or "Mayo” effect. And, experi- 
ments \nlhin one school, conducted 
with lime as the control variable, often 
stimulate other kinds of change within 
the school — even to the extent of draw- 
ing a different applicant-pool of stu- 
dents. 

As if these difficulties were not 
enough, another — less clear — problem 
Tweeds mentioning* Each instructional 
mode may In f.irt have different goals 
which can IksI be served. Thus, to at- 
tempt a comparison lu achicvcmciil 
when different teaching methods arc 
used might start with a major bias if the 
achievement of the same educational 
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objectives is attempted using methods 
truly adaptable to quite divergent goals. 
But at this stage, such a consideration 
may be less important. It is presently in 
the best interest of medical education 
and of the use of computers that experi- 
mental design be accomplished wherev- 
er humanly possible. 

\Vhere experimental design is not fea- 
sible, however, there is still the need for 
good, thorough evaluation. Has the edu- 
cational experience really brought about 
the changes in students that were de- 
sired? (Were the objectives realized?) 
How much faculty time and effort were 
required? (How much did it cost?) 
Such an approach required “only'* the 
definition of objectives, the statement of 
criteria, the selection or construction of 
measurement devices, the performance 
of measurement, and the interpretation 
of data collected. (Kote the setting off 
of the word, “only'’!!) If this process is 
achieved properly, the investigators will 
at least know whether objectives have 
bc^'n realized even if they cannot tell 
whether the process represents some- 
thing “belter" than another approach. 

The role of the innovator is not an 
easy one. Not only is he the guy with 
the "ideas." Not only must he then find 



ways of accomplishing his ideas. Not 
only does he become the activist in in- 
troducing the innovation. Not only must 
he seek and obtain sup;x)rt for his idea. 
He must also test the idea and be pre- 
pared to defend — with data - in the 
face of all the difficulties in obtaining 
such data, his innovation. The role of 
the true innovator is a demanding one. 
He must be imaginative and flexible in 
order not to become 'locked in" to one 
approach in the application of his genre 
or innovation. Computer use is a case- 
in-point. Storage and retrieval of infor- 
mation is but one of the many potential 
applications, yet for many this has be- 
come the end-all. He must promote the 
search for evaluative data — despite the 
fact that there arc no such data avail- 
able concerning conventional modes of 
teaching. He must serve as "public rela- 
tions" man for the entire area of innova- 
tion — not as high-pressure salesman, in- 
cidentally, but as thoughtful, dala-ladcn 
investigator, raising questions and pro- 
viding data rather than exhorting, pro- 
moting, and promising Utopia through 
panacea. Hero, then, Is the essential con- 
tribution of evaluation of the use of 
computers in medical education, 
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EDUCOM— A PROGRESS REPORT 

JAMES G. MILI.ER. M.D., Ph.D.* 



EDUCOM, the Intcrunivcrsily 
Communications Council, is a proto- 
type endeavor, a rcscarch-oricntcd at- 
tempt to bring about educational in- 
novation. Let me begin by illustrat- 
ing, in a somewhat lighthearted and 
irrelevant fashion, the problems of in- 
novation and the need to move slowly 
and cautiously when initiating new 
programs. 

A pretest had been carried out on 
the effects of diet on the aging proc- 
ess. The experirricntal animals were 
porpoises. The porpoise is considered 
by mj^ to be nearly as intelligent as 
man. They have large brains and arc, 
in a sense, aquatic, humanoid bashet- 
cases. The porpoises were given a 
number of different feeding regimes. 

A diet of newly hatched sea gulls was 
found to slow down, almost to halt, 
the aging process. It appeared that 
what amounted to immortality might 
lx? a possibility. 

The pretests were so striking that 
tlic research group received a large 
Nil! program-project grant. I^irgc, 
concrete jwrpoisc pens were built on 
Ibe seashore for containing the cxjX’r* 
imcntal animals. Up in the hills, an 
aviary for hatching sea gulls was 
built. A tnick was di^atehed daily to 
pick up the young fledglings and to 
take them dowTi to the pens to feed to 
the porpoises. 

Chic day as the truck passed the 
City Zoo on its regular run, an old, 
distinguished, sedate lion suddenly 
and inexplicably botindcd out of his 
cage and Into the path of the truck. 

The truck could not help but run over 
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him. The police soon arrived and ar- 
rested the truck driv er for ‘"transport- 
ing young gulls across staid lions for 
immortal porpoises.’^ If this irrelevant 
and irreverent fable does not illus- 
trate the dangers of innovation, noth- 
ing can. 

EDUCOM was founded about 
three years ago by a numlx?r of medi- 
cal deans ancf chancellors, specifically, 
those at Duke, Virginia, State Univer- 
sity of New York, Rochester, Pitts- 
burgh, Michigan, Illinois, and Cali- 
fornia. It was founded to facilitate 
and accelerate the use of the new in- 
formatiori processing technologies, 
where desirable, throughout the 
health sciences, educational, research 
and service activities of colleges and 
universities. 

It soon became apparent that ED- 
UCOM's functions should not be re- 
stricted to the health fields but 
should be extended to all academic 
disciplines. While some of EDU- 
COM s present members of the Roard 
and officers arc physici.ins, the major- 
ity arc not. The organization has ex- 
panded rather rapidly although very 
little effort has lx?cn made to give it 
pul)licity. All article in SficrifC ap- 
jx’aring in October, 1966, is the only 
published article describing the total 
EDUCOM program. 1 lie re have, 
however, Ix'cn a number of publica- 
tions relating to various sjvccific proj- 
ects. 

At the moment, there arc S9 univxr* 
silies on our membership roll. They 
include alxiut 2-50 campuses. EDU* 
COM is OiKn to all colleges and unb 

Treiic/eht, The Cteedfind Siote Vnkersity. 
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vcrsilics of Canada, the United 
States, and Mexico and will be 
pleased to accej^t your institution into 
membership if it is not a member. 

Several universities in England, 
France, Czechoslovakia, Sweden, 
Holland, and Australia have indicat- 
ed an interest in can)ing out similar 
activities and in cooperating \nth 
American universities. We intend to 
consider potential international appli- 
eations as soon as some of our initial 
North American projects have given 
rise to serviees which will be of real 
help to participating institutions. Im 
ternational cooperation across oceans 
will be facilitated by edueational sa- 
tcllites which may be in orbit four or 
five years from now'. 

Dr. Jordan Baruch has recently be- 
con'jc our new President. A member 
of the MIT staff, he was formerly the 
head of the MEDINET program of 
General Elcctrie. With the transfer of 
our national headquarters to Cam- 
bridge, Massachusetts, we have de- 
veloped a redefinition of our func- 
tion. In the words of our new presi- 
dent, a major HDUCOM func’tion is 
**10 facilitate the exlraorganizational 
communication of the universities.’* 
The reason that the emphasis is on 
extraOTganizational communication is 
l>ccausc HDUCOM docs not, except 
indirectly, intend to p.articipale in ac- 
li\*ilies which can be carried on as 
well by an individual professor or an 
individual university. \Vc arc interest- 
ed, rather, in helping institutions of 
higher education to share their re- 
sources and carry out joint programs 
which can be more efficiently han- 
dled, improve quality, and/or cut 
costs. We arc concerned with rela- 
tionships not only among universities 
but between them and other parts of 
our society. Wc shall bo concerned 
with such extraorganirational activi- 



ties as the regional medical programs, 
for example, which involve not only 
universities and medical schools, but 
also private hospitals and private 
practitioners. 

PANELS ON RESOURCE 
DEVELOPMENT 

EDUCOM is presently involved in 
reorganizing its technical and re- 
search programs, forming a new se- 
ries of national Intcruniversily pancls. 
In the past wc have operated tnrougli 
intcruniversity task forces. The first 
task force was responsible for the 
programs in nctworwng. The chair- 
man of this task force has been Dr. 
George W. Browm, who was the 
founder of the Western Data Process- 
ing Center at UCLA and who has re- 
cently become Dean of the College of 
Administration at the University of 
California at Irvine. 

The original task forces were 
“bootstrap” operations. Volunteer 
w’orkers irom the participant universi- 
ties gave their time, prepared pro- 
grams and proposals, and carried out 
research activities. EDUCOM has 
now attained some financial stability, 
however, having received several 
foundation grants, industrial grants, 
grants or contracts from the federal 
overnment, as well as institutional 
lies. It will soon be feasible for each 
of the panels to have full-time one or 
more professional staff workers, ei- 
ther in the central office in Cam- 
bridge or in the Bcthosda office, 
where some of the gDvcmmcnl sup- 
|X)rtcd research activities are now- 
going on. 

The network task force vv'jl be 
reorgani/e<l into the panel on Inter- 
connected computers, which will be 
concerned witn the organizational 
and structural interconnection of 
computers, the hierarchical organlza- 
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lion of data within them, and human 
access lo ihcir caj^abililics. 

A second panel on resource devel- 
opment will be the panel on video 
and films, concerned with the use of 
\-ideo recordings, films, live produc- 
tion, closed circuit TV, and other 
such tools for augmenting the educa- 
tional process. 

Third, the panel on mierographic 
dcN^clopment will be concerned with 
the utilization of such techniques as 
pholochiomic reproduetion in storing 
masses of document information, the 
interphase between sucli systems and 
the computer, and the use of such mi- 
crograpnic images. 

The U^or^; of a Panel 

A new panel goes through a stage 
of group dymamies and a stage of 
self-definition of their mission. It may 
Identify areas of appropriate activity 
for EDUCOM and it can suggest spe- 
cific projects for EDUCOM or some 
subset of Its members. Such a panel 
will be different from a sludv section 
of tbc National Institutes of Mealtb, 
for example. In that its major role is 
lo initiate programs rather Inan lo re- 
spond passively to programs suggest- 
ed by individual professors or by uni- 
versities. Rut it certainly can receive 
sueb suggestions. In a few cases such 
suggestions have eventuated in pro- 
grams that have been supported or 
sponsored by EDUCOM. 

EDUCOM panels can communi- 
cate among ibcmselvcs lo establish 
joint aclivilies, since there will he in- 
Itrscclions of interests among the 
panels. And they can maintain tempo- 
ral continuity. The life span of a 
CTOup or organization may l>c far 
longer than any Individual human life 
span, ^rhis means that continuity of 
programming Is feasible even if indi- 
nduals rotate In and out of various 
paneli. 



Panels (n the Applications and Uiilu 
zation Area 

The first of these is the panel on 
extended education, concerned with 
the continuing education of the 
professional, with the resumed educa- 
tion of those whose education has 
been interrupted, and with (he reedu- 
cation cf those dissatisfied with their 
careers. The panel on extended edu- 
cation is a reformulation of the task 
force on continuing education, 
chaired by Dr. George E. Miller, Di- 
rector of tbc Office of Research In 
Medical Education at the University 
of Illinois College of Medicine. 

The second panel in the applica- 
tions and utilization area is tbc panel 
on libraries and data banks. The pan- 
el is concerned with the utilization of 
stored information in the educational 
proct'ss, UTth the instructional impli- 
cations of the fact that large bodies of 
data can be made rapidly accessible, 
and with the development of the li- 
brary function broadly defined. The 
work in library communications was 
formerly in the EDUCOM task force 
on networks. It has become clear to 
us that, while there have boon and 
will continue lo be important over- 
laps Ik I ween networking actMly and 
library functions, they should be 
dealt with by separate panels because 
they arc fundamentally independent 
fields. 

The third EDUCOM panel on ap- 
plications ai d utilization is the panel 
on external affairs, concerned with 
legislative matters, public policy, and 
sodal issues involved in cduealional 
Information processing. Its predeces- 
sor was the EDUCOM task force on 
legal and related mailers. It is co- 
chalred by Professor Benjamin Kap- 
lan of the Iharvard Law School and 
Professor Arthur Stiller of the Univer- 
sity of Michigan Law School. The 
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new panel will include allomeys, spe- 
cialists in libraries^ television, and 
computers, and educators. EDUCOM 
has taken an active role in a number 
of matters directly related to utiliza- 
tion of the new technologies in edu- 
cation, of whose importance many 
university faculties and administra- 
tions are unaware. For example. Con- 
gress is presently working on new 
copyright legislation, the first to ap- 
pear before Congnss in alxiut &) 
years, The draft of the bill passed by 
the House precluded the possibility 
of operating legally such an intcr- 
imivcrsily computer, library or televi- 
sion network as EDUCONf envisions. 
ED?JCONf uijiversilics and officers 
sent statements or appeared at the 
Senate Committee hearings. These 
matters were also discussed with rep- 
resentatives of publishers and other 
industries. EDUCONf has had some 
role in slowing action on the copy- 
right legislation, which was written 
before tne importance and implica- 
tions of computers and nctworli be- 
came clear. The legislation can now 
be rewritten to permit adjustments 
for new technological developments 
as they occur, 

EDUCOM is also concerned \sith 
antitrust and negligence laws in the 
operation of intcruniversity and in- 
Lrauniversity networks. A nonlcgal 
but very important area concerns Inc 
ethical use of confidential informa- 
tion. This Is particularly important to 
physicians in terms of patient records. 

EDUCOM S Plans for S^ctworks 

Although EDUCONf Is concerned 
with all applications of technologies 
in education, it has dc\x»lcd particu- 
lar attention to the sharing of re- 
sources through the dcNrlopmcnt of 
nct>^c)rks, planning to develop and 
operate a prototype intcruniversity 



network. Networks arc nothing new. 
Natural netw’orks have been in opera- 
tion as long as there have been col- 
leges and universities w^ith one-way 
or two-way communication between 
two or more students and faculty 
members. A conference is one form of 
operation of a natural nehvork. 

A different kind of network is a 
physical network like the telephone 
network or the mails. A thorough 
analysis of communication among 
and w^ithin universities requires a 
comparative evaluation of traditional 
and new networks. Because a net- 
work employs a new technology ob- 
viously docs not mean, of course, that 
it is better. 

At the same time that EDUCONf is 
involved in planning clcelronie net- 
works, it is also developing an organi- 
zational network. JJacfi of our mem- 
\x!T universities has an institutional 
representative who communicates 
with his campus and represents his 
iinbcrsily with a vote on the EDU- 
CONf Council. The Council elects the 
Board of Trustees and the Board of 
Trustees selects the permanent offi- 
cers and full-time staff, ff wc place a 
prototype network into operation, it 
must have its own organizational 
structure as well. 

EDVSET 

The most Intensive planning activi- 
ty related to networks in EDUCONf 
was accomplished at the University 
of Colorado at Boulder in ]ulv, 1966. 
Tlie conference was fnndca by a 
grant from the Natioral Library of 
Nicdicino, the National Institutes of 
Hralth, the National Science Founda- 
tion, and the Office of Education. 
Alx)ut 1S5 people attcrxled, repre- 
senting each or the 42 univxrsitics 
then members of EDUCONf, the ma- 
jor long line arKl hardware compa- 
nies, publishers, approxirnatcly 15 
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government agencies, and a number 
of foundations. The conference prod- 
uced a book called E DU NET, a re- 
port of the conference including opin- 
ions of llic participants and a set of 
recommendations 

The conference almost unanimous- 
ly agreed that there were real needs 
in colleges and universities for an in- 
tern ni vers ity nctwork- 

It was unanimously agreed that 
many different t\pes of networks are 
technically feasible. An analysis was 
made of different possible configura- 
tions of networks, and their possible 
costs were estimated. The book also 
outlines an organirational structure 
which would be able to operate such 
a network. The last chapter Is an in- 
formal projwsal to a number of gov- 
ernment agencies who might contrib- 
ute to the funding of such a prolo- 
t)po. 

The basic concept of EDUNET is 
to encompass a number of activities 
already underway in a limited fash- 
ion, Some of these arc limited geo- 
graphically, regional networks. EDU- 
NET is conceived as a national or 
conceivably, ultimately, an interna- 
tional network. Some networks are 
limited to a single medium, c. g., a 
network for programmed instruction, 
A television network, or a network for 
tclet)^ transmission of bibliOCTaphi- 
cal fniormation. EDUNET is devised 
on the assumption that all media 
must be employed, each when most 
appropriate educationally. From the 
It'chnical jx)iul of view, it is possible 
to o^Kratc multiple media on the 
same bandwidth, switching from one 
to another as reejuired. EuUNET is 
also intended to In\X)lvc all disdplincs 
within the university. Some of EDU- 
COM's projects are for one field, such 
as ovir plans for networks in bioagri- 
culbirc and biomedicine. Such en- 



deavors are useful ways to begin, but 
limitation to a few disciplines is edu- 
cationally unsound. Other fields 
would feel neglected if only physi- 
cists, chemists, engineers, and physi- 
cians had access to the new forms of 
information processing. EDUNET is 
also devoted to achieving as soon as 
possible on-demand access by the user 
to information, on-demand access, not 
only to documents but to television 
programs, computerized programmed 
instruction, anu all other media. 

We propose to join a number of 
colleges and universities in an experi- 
ment lasting three years or more — 
hut for a definitely limited [ioriod. 
This experiment would be designed 
to indicate what is feasible and what 
is not, what is worth doing and what 
is not, what is loo costly and what Is 
reasonable in terms of dollars and hu- 
man effort. The network would be 
operated with one staff and evaluated 
with another, to be sure that the op- 
erators do not bias the evaluation of 
the benefits the system provides to 
the users. 

We propose to establish an EDU- 
NET lalx)ralory with a single director 
supervising a staff at three or four lo- 
cations, One location would be at the 
Research Triangle in North Carolina, 
one In Ann Arlx)r, Michigan, and one 
on one of the southern campuses of 
the Universiiv of California. A fourth 
location inignt be at an Ivy league 
university. The EDUNET branch 
lalx)ratorics vvovjld bo interconnected 
with fovir switching stations, jx)ssibly 
at San Francisco, Denver, Chicago, 
and Boston. The entire interconnec- 
tion would constitute the kernel nel- 
worV. 

The first year wo would spend ap- 
proximately a million dollars recruit- 
ing a staff and deciding on such ques- 
liorvs as the organizational slrudure 
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for EDUNET, its opcratir*;; policies, 
issues of compatibility for hardware 
and software, the selection of hard- 
ware for the branch laboratories, 
what materials would be put on the 
network, the selection or long-line 
services, and how costs would be ac- 
counted for and charged to users. In 
the second year, budgeted at approxi- 
mately three million dollars, narrow- 
band interconnection of the |X)inls on 
the kernel network \\ouId be estab- 
lished and one after another sen'icc 
would be put on it. By ""narrow-band** 
we mean a bandwidth equal to ap- 
proximately 12 parallel telephone 
channels. 

WTiencver tmy service became op- 
erative, any EDUCOM university 
that wanteef to use it could do so, It 
would have to pay the long-line 
charges to the nearest jx)int on the 
kernel network, The university could 
use the service iny way it wished to 
on its own campus. If it had a remote 
terminal system or computer system 
which was incompatible wilh EDU- 
NET, it would nave to make the 
transformations required — cither 
through a black box or V)y program- 
ming. Conversely, if they wanted to 
contribute a service prepared on their 
campus to other universities through 
EDUNET, they would have to put 
the infonnation into EOCCOMpati- 
ble formal so that it could be carried 
on the network. 

In the third year, we v ould expand 
the bandwidth to permit two-way 
color televisian. Tlic cost would Ik* 
about six million dollars. Wo may lx? 
underestimating costs. That depends 
on how complex our services iKcomc, 
unanticipated operational problems, 
and inHation. Wc arc attempting, 
however, to keep the costs as low as 
wc possibly can so that there is a rea- 
sonable chance to raise the necessary 



fvmds. The universities will contribute 
their appropriate share of the funds 
rcfjuired. The governmcjit will not be 
asked to provide the total amount. 
The universities will pay for their 
own terminals, for the long-line 
charges to interconnect with the ker- 
nel network, and for some services. 

Many of the services wc pul on the 
network will be prepared by govern- 
mental or nonprofit priva^e organiza- 
tions, Wc have made tentative ar- 
rangements wilh quite a number of 
such organizations to use their serv- 
ices. Such inputs will of course be 
much less expensive to us than if we 
had to prepare them. A considerable 
amount of machine-readable informa- 
tion is available now to go on such a 
network if arrangements c.in be 
made. 

Tl)c Office of Science and Tcch- 
nolo ,y has taken cognizance of our 
pro'^: /sal, has assigned responsibility 
for t to the Department of Health, 
Education, and Welfare, and under 
HEW to Dr. Ix)uis Bright, Associate 
Commissioner for Research in the Of- 
fice of Education. On various occa- 
sions ho has called together interested 
agencies to discuss the EDUNET 
pro|X)sal. 

It seems probable that EDUCOM 
may bo requested to extend the ]>lans 
for another less ambitious network. 
\\*c hast? proposed EIN, the Educa- 
tional Information Network, to in- 
clude certain ns|Kcl5 of EDUNET. 

We have had a discussion al>out 
EDUNET uith a numlKT of the 
Wliite House staff. On No\cml>cr 7, 
19G7, the President announced his 
siipiaorl for the desclopmcnl of "net- 
\\xtrV% for knowledge". In the Presi- 
dents message on education to the 
1968 Congress, ho recommended the 
Networks for Knowledge Act of 1968, 
which is Title 9 of the Eslucation Bill 



1 -f n. R. 15067. ITiis proposes 
networks almost i(!cntical in principle 
to that outlined in the book “EDU- 
NIC JV’ Healings on this bill were held 
several weeks ago by the Special 
Sub comm iltee on Education of the 
House of Representatives and the bill 
is now being re\'icwcd in the Senate. 

I have mentioned EIN, the Eduea- 
lioiiul In format ion Network. EDU- 
(COM is shiving for the ultimate de- 
velopmu^t of nil embracing program 
like that of ICOUNET. At the same 
lime it is important, however, to carry 
on networking activities relevant to 
llic missions of various funding agen- 
cies. We have a contract with the Na- 
tional labrary of Medicine to plan a 
National Riomcdical Communications 
Center which may involve network- 
ing in tlial field. \Ve have a contract 
with the National Library of Agricul- 
ture to develop a plan, in association 
with tlic EDl'COM landgranl univer- 
sities, for infoiinatioii transmission in 
Lioagriculture. We have an agree- 
ment with the Dental Health Center 
of the Cniled States, when funding is 
available, to work in the area of a 
dviital lieaUli information system. We 
li.ivc a gr.iiil under eonsidcralion by 
the Ofliee of EdvieatiOn jointly with 
the National Science Foundation for 
t/.e cU\elopmcnt of EIN, a network 
limited to one cf the media, i.c., com- 
putus, \N oik cm iho Educational Iii- 
fonn.duni Network is expected soon 
to begin acx-ording to the following 
planm d chronology: 

Phase I 

1. Engage a limited group (up to 
i^O) uaiuTsilies in agreement |Krinit* 
ting icnv)te «icxtssing of their coni- 
I nil r ficiliiie f. (More than 20 univer- 
sities h.r\c already volunteered to 
lo-jpi rate.) 

2. Estaldish standard documenta- 
tion fornial for a description of com- 



putational facilities available and 
produce a directory of such computa- 
tional facilities for participants. 

3. Establish tcennieal representa- 
tives at each participating location to 
act as liaison and sources of informa- 
tion about operations. 

4. Set up a record-keeping system 
to identify usage patterns for charges 
and further planning. 

5. Establish an on-line directory to 
supplement the initial printed direc- 
tory. 

Phase II 

6. Establish a means for data trans- 
mission among centers consistent 
with their individual format con- 
straints. 

7. Develop a system of automatic 
referencing Ix^lwcen and among com- 
puters at participating facilities, and 

8. Generate and docnmcnl data 
bases for expanded nets. 

We believe that many man-years 
are being spent in different EDU- 
COM universities In WTiling pro- 
grams for local computers which arc 
already or computers in other univer- 
sities. Iliis is true of programs of 
many sorts - for programmed in- 
struction, for data banks, and for 
computational use of computers, 
among others. To diminish the great 
costs of such duplication, EIN may 
enable a user on one campus to dial 
Into a c<impulcr on another campus 
which already has the program he de- 
sires, so pcnnilling him to benefit 
from the work already done else- 
where. 

I have been able to mention only 
some of our present programs, and 
those only briefly. We expect that 
there will be applications for such 
programs to the health fields, to pa- 
licnl-rccord $)*slems and to medical 
education, research, and scnicc. It Is 
vital that all sdcnlisls and profession- 
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al people be able to gain rapid access 
to relevant information and that our 
institutions be able to share r( sources, 
Xelworks can help in this, lliey can 
enable our universities to become a 
society of universities, to establish in 
our nation universities without walls 
that permeate many actmties of our 
country. They can give us democracy 
of access to information. Not only in 
Boston, New York, Chicago, and San 
Francisco will users be able to obtain 
specialized information in depth, as is 
now true, but a new college or uni- 
versity wherever it is, a small institu- 
tion in the Rocky Mountains or a fi- 
nancially weak Negro college in Ala- 
bama can have as good or lx ttcr ac- 
cess to information as Harvard docs. 
Networks can aid in improving the 
general quality of instruction, re- 
search, and professional s c r \ Ices 
tliroughout the nation. 

Disettuion Periods . 

Question: Has FDUCON! devel- 
oped relationships with incliivt/y? 

Dr. Midrr: \Vc I lavc received no- 
strings-allached grants for FDUCOM 
activities from llie General Lc.i ruing 
Coijwralion from IBM, and in snial- 
h r amounts, from other ct)rjxjf,Uions. 
W’e a 1.0 have received a grant from 
HCA to support a con fere i>ee. \\V 
have six membcrs-al-largc on the 
Board of FDUCOM and llncc of 
those inembcrs arc from industry. 
Kach of them is an individual vvho is 
distinguished in education, for exarn- 
le, Francis KIcppel, vvho is well 
nown as an cduc«\lor and is also 
Chairman and rrcsidcnl of General 
learning Corjx>ralion; and Carrol b 
Newsom, Vice President for Fdvi ca- 
tion of RCA. former President of New 
York UniuTsily, We have also a I 
present a representative from IBM 
and one from General Klcctrlc. At 
variotis EOl'f'OM ct>nffrcnccs repre- 



sentatives of the publishing, long line, 
hardware, and broadcasting indus- 
tries, have participated. While KDU- 
COM is planning a prototype experi- 
mental network, it is not at all sure 
that in the long run it is the responsi- 
bility of the universities to run such 
networks. After vve arc clearer how 
network serv’ices can help universi- 
ties, some industry or indusaics may 
wish to provide the scrv'iccs on a rea- 
sonable profit-making basis. That 
would be qiv'te acceptable to EDU- 
COM. 

Has EDUCOM devoted 
attention to problems of compatibility 
among local and regional networks? 

Dr. SfiUcn Wc may expand the 
KIN study to determine the technical 
characteristics of all the regional edu- 
cational networks, for computtrs, li- 
braries, television, and so on, in an e f- 
fort to ascertain to what extent (hey 
arc compatible. Millions of dollars of 
loc’ol, state, and federal funds are go- 
ing into regional networks of various 
sorts. So far as 1 know, no one region- 
al network planning group is taking 
informed notiex’ of llic technical char- 
acteristic's of another in orelcr to be 
compatible, l/DUCOM may attempt 
first of all to learn the characteristic s 
of present networks. Then, when vve 
have developed, in the light of these 
facts, .siigge^stions alxuit v\hal riiighl 
l>e appropri.Uc standards, vve vvill 
projxjsc them through proper techni- 
cal channels, Wc arc conscious of tlic 
fact that if )oii cslalilish compitibili- 
ty regulations loo soon vou may re- 
strict proiKf elevilopmrnt On the 
other hand, sometimes decisions imtsl 
bo made or chaotic waste exxurs. 1 
[KTSonally Wlicsc wc are near the 
lime when vve must make such deci- 
sions. Wc have talked to llio Hurrau 
of Standards aloul such matters, as 
well as to other agemhs involved in 



31 



35 



setling standards. While some stand- 
ards — for television, for example — 
are now well established, in other 
areas there are none at all. We would 
obviously follow Library of Congress 
or Library of Medicine standards or 
others set by responsible bodies 
whenever they exist. In areas where 
standards have not been formulated, 
some group must make the decision. 
Large hardware and software invest- 
ments arc being made by universities 
in the current year. If these products 



obtained are so incompatible that 
Wisconsin cannot take advantage of 
developments at Cornell or Cleveland 
State cannot benefit from work at 
California, there will be a great loss 
to all of us. There is danger of a de- 
bacle like the color television hassle 
between CBS and NBC some years 
ago or the battle among the makers of 
different sizes of Iong*p!ay records or 
tape cartridges. A solution must be 
found soon and EDUCO.M is aware 
of the problem. 





COMPUTERS IN UNDERGRADUATE 
MEDICAL EDUCATION 

HILLIARD JASON, M.D., ED D.* 



Undergraduate medical education in 
this country is in a state of re-examina- 
tion and revision in proportions without 
precedent. '‘Efficiency’’ and “effective- 
ness” have become watchwords of cur- 
riculum committees. Patterns and tech- 
niques of instruction which had been 
hallowed by decades of use arc being 
modified and even jettisoned, and the 
search is on for new instructional modal- 
ities which hold thn promise of getting 
more done in less lime. 

The growing sense of need for Wttcr 
educational methods has been parallel cxl 
by an increasing availability of potential 
solutions. Within the past decade sve 
have come to understand the process of 
instruction more precisely and techno- 
logical aids to the process have ap- 
peared in increasing prohisjon. Pro- 
grammed instruction, teaching ma- 
chines, automatic cartridge movie pro- 
jectors. simplified audio recorders and 
television arc all part of the growing 
technology of education, Pecan sc of its 
far greater flexibility, jx>tcncy and cost, 
the computer represents the ailmination 
of this trend. 

To understand the promise and the 
problems computers might bring to 
medical cdiiealion, vv*e must tmdcrsland 
something of; (1) the process of intro- 
ducing a major innovation into an edu- 
cational S)*stcm, (2) the current educa- 
tional requirements in medicine, and 
(3) the particular characteristics of the 



computer which may or may not fit into 
these educational requirements. 

/nnotxifiou <n Ecfucutlon 

An innovative device of the scope of 
the computer can be a mixed blessing. 
We have recently begun to enter a new 
era in medical education: one in which 
a half century of educational conserva- 
tism is giving way to a receptivity for, 
indeed a hunger for, new and promising 
approaches. As much as many of us re- 
joice at this venturesome spirit, vv*c must 
j)C wary of its potential pitfalls. Most 
worrisome of all is the capacity of new 
dciiccs to be mistaken for new ideas. 
The act of accommodating old instruc- 
tion to new packages, sucli as television 
or the computer, can prove to be worse 
than mere maintcnanct' of the stains 
quo. The glossy new packaging lends to 
deceive l>oth the sluticnl and the Ic.icb- 
cr into l>clieving th.vt soinelbing new, 
and therefore belter, is happening. In 
actual fact, what often happens is that 
considerable energy and money are in- 
vested in creating a situation that is 
even harder to change than vvas the 
conventional instruction it is mimicking. 

Wc have rcjKalcdly ^ren, for exam- 
ple, the poorly delivered slultif) mg lec- 
ture become even more so on the T\^ 
monitor. With sadness I must report that 
the equivalent has begun to occur with 
the computer, in some people’s hands. 
One can now find examples of electronic 
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reincarnations of the pompous, arrogant 
professor whose instructional strategy is 
the demand of compliance rather than 
the promotion of independent skills in 
the student. The computer generated 
rejuimand of ‘‘Nonsense! You should 
know helterT' fails to help the student 
learn what he did wrong or where to go 
next. Simply puti bad instruction isn't 
made good by inclusion in a new medi- 
um. The essence of these miscarriages of 
modern methods is that they are at- 
tempts to provide solutions, unthout 
awareness of the nature of the existing 
problems; they arc accommodations of 
conventional instruction to available 
technology, when continuing modifica- 
tion of the inslniction itself and adapta- 
tions of the technology arc ncctlcd. For 
cducalion.il ijinovalion to be meaning- 
ful, it mvist derive from a sound \indcr- 
standing of educational rcfpiircmcnts, so 
that available tc chnology can be proj>cr- 
ly modified and exploited, and thereby 
proude worthy solutions. 

Requirements of Mcr^ieal Education 
A. What We fx^arn 

I will turn now to some rcrjuircmcnls 
of an effective medical education: first, 
in terms of what it Is that students must 
learn. 

The most frequent and most desper- 
ate plea one hears today in medicine is 
that we must find ways for dealing with 
the inundation with which we arc al- 
lege<lly faced, as a consequence of the 
“informaiion explosion,” Statistics arc rc- 
iicatedly cited on the mviltfplying num- 
!>cr of journals and journal articles as ev- 
idence of this trend. There arc two sides 
to this problem which deserx e our atten- 
tion. They arc neatly highlighted by the 
following quotation: 

“Tliosc who have oceasion to enter 
into the depths of w Iial is oddlv, if gen- 
erously, called the literature of a scien- 
tific subject, alone know the difficulty of 



emerging with an unsoured disposition. 
The multitudinous facts presented by 
each corner of Nature form, in larg? 
part, the scientific man’s burden today, 
and restrict him more and more, to a 
narrower and narrower specialism. But 
that is not the whole of his burden. 
Nfuch that he is forced to read consists 
of records of defective experiments, con- 
fused statements of results, wearisome 
descriptions of detail, and unnecessarily 
protracted discussions of u inccessary 
h>T>olheses, The pubh'catioi of such 
matter is a serious injury to the man of 
science.” 

These timely remarks arc taken from 
John I^anglcy's presidential address to 
the Physiology Section of the Royal So- 
ciety, in IS99 (I). It seems that the in- 
formalion explosion, and its accompan)*- 
ing “garbage explosion,” began some 
time prior to the present decade. 

Most of iis would willingly concede 
that much of the seeming explosion of 
knowledge is in truth, a dilution with 
triWa and irrclcxancies. There has been, 
howevrr, an undeniable and \cry «5n- 
sideraMc cxj>ansion of our \indcrs land- 
ing of life processes. 1 ebalirnge, howev- 
er, the \isual implicalions that are de- 
rived from this expansion. Rather than 
complicating tl;c process of learning. I 
contend that the grow th of knowledge 
has actually simplified the process, f 
will turn to some other quotes to help 
make the point. 

“I know that \ a n shockingly igno- 
rant. I could take exams in college luit 
eo\iTd not pass any of them. Worse than 
that; \ treasure iny ignorance; 1 feel 
snug in it. It docs not cloud my naivete, 
my simplicity of mind.” 

This is not the statement of a San 
Francisco "flower chiUI it is the obser- 
vation of one of our most distinguished 
scientists, the Nobel laureate, All>crt 
S/ent Cyorgyi (2). It was said in a con- 
text that should l>ccomc clear from the 
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following additional quotes; 

‘It is widely spread opinion that me- 
morizing will not hurt, that knowledge 
does no harm. I am afraid it may, Dead 
knowledge dulls the spirit; fills the 
stomach without nourishing the body.” 

He points out that we have books and 
libraries **lo keep the knowledge in 
while we use our heads for something 
belter.” And it is our instructional chal- 
lenge to help students learn to use their 
heads for ‘'something better” than sim- 
ply storing information. A major point of 
this paper, to which I will shortly re- 
turn, is that the computer can help us do 
better jobs of both storing information 
and teaching students to use their minds 
for “something better.” First, let me lin- 
ger one moment longer with Szent- 
Gyorg>i, to emphasize a notion that will 
help clarify the point. 

"Science tends to generalize, and gen- 
eralization inc.ins simplification. My 
own science, biology, is today not only 
very much richer than it was in my stu- 
dent days, but is .simpler, loo. Then it 
was horribly complex, being fragmented 
into a great numl>cr of isolated princi- 
ples. Today these arc all fused into one 
single complex, with the atomic model 
in its center. Cosmology, quantum me- 
chanics, DNA and genetics, arc all, more 
or less, parts of one and the same storv 
— a most wondrous simplification. And 
generalizations arc also more sati^f)ing 
to the mind than details. \Vc, in our 
leaching, should place more emphasis 
on generalizations than on details.” 

Tut in other words, the expansion of 
knowictige lias left us with a lighter, 
rather than a heavier, burden. \Vc are 
far l etter equipjx^ than ever Ixforc tu 
help our students come rapidly to grips 
with the central, organizing issues of our 
disciplines. \Vc can preside the concep- 
tual base which pernuts the skills of the 
rminiptddfion of knowledge, the skills of 
acquiring, validating and using knowl- 



edge, rather than the mere possession of 
knowledge, to become the mark of the 
educated man in medicine. It is the acts 
of evaluating and gcmTjting kiiowb dge. 
more than retaining it, that scern proper 
goals for our inslnictionrd ]iro':;rajns. 
And it is in this transformation of gnnls. 
not in the adoption of ne w* \ irr^, that 
the heart of edueationa! iiniov.itiou lies. 

B. How’ \Vc Ix*arn 

Ix*l us turn from u hnf stude nts dimiM 
learn to how they ham. .Aiming the 
many conditions of (Ihctivc loaniing. 
two requirements are ]vnvinonntr I’ii'^t. 
the student must pr.iclire, (/o rig h 
ing, the very things he is (o lu^ abk* to 
do as a result of learning. Tlv I is. if hr- 
is to be able In distinguish abnompd 
from normal heart sou mb, he inn'll nr.u ■ 
lice that ver)" act. H he is (o be d le to 
.solve elinicnl i^rohlorns, hf‘ nniM praf 
the various slips, the (onqiom nis, <»f 
problem solving. S( ond, the Mml 
must hav e frequent and n h d k 
mation on ibc appropt iiii iv * s the 
]UOgress he is inakiug: he mn^t l>a\r 
"feedhaek.” To illustn i(n if hr is to 
learn to sw ing a golf Huh. h ' mod not 
only get to see win re thr hill <n>rs a 
result of Ills efforts; ho will mike hi^ 
most efficient ]>rogn if lu' ran m <’ a 
mo\ic or videotape of hi> r.eto ,1 >v 'n g 
am! he is helped to s< (' dij r.nr' d com 
poncnls of the lot.il skill. m;Hi th ’ yi 
sition of his head, the hur nf l>r«r •-Voul 
ders. hi« stance, and ^o r>n, SiMvlnlv. if 
the student is to Irani to «.o!\e nii dir d 
problems, he nerds to know the raurrd- 
ness of liis solutions but bo abo nri d^ 
to know bis coTiipHrmo at m an.igMv: 
the crucial coirnx>ncnts of llic pioc^ s'; 
sucl as problem scnnYig. piobk ni for- 
mulation, information s^arHi ard rc 
lion. Those skills hasc been idontiiied 
and clal>otalcd ly Shnlr vui (3b 

While the earning of inlcllrclual dills, 
such as i^roblcm solving, is fir norr 
complex than the learning of mcch u’cal 
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skills such as golfing, wc liav^ been con* 
sidcrably less effective than golf instruc- 
tors in providing the exquisitely impor- 
tant ingredient of meaningful feedback. 

To recapitulate, I suggest that among 
the central challenges of current medical 
education are: ( 1 ) developing tech- 
niques for helping students learn to ef- 
fectively manipulate knowledge, (2) en- 
abling them rcix^atcdly to practice these 
manipulations, and (3) providing fre- 
quent and relial>Ic feedback on their 
success at the overall process, and at its 
components. Computers have a unique 
set of characteristics which, if properly 
exploited, can fulfill these requirements 
with remarkable effectiveness. 

Characteristics of Computers 

The specific characteristics of modem 
C'omputers and the ways in which they 
function, have been well described in a 
recent pviblication by Caffrey (-1), in- 
tended primarily for technically unin- 
formed readers. The gcne.’al characteris- 
tics with which I will here deal are 
those matters which proridc Ithe conr- 
puter with its special ^wtcntial as well as 
its limitations, as an educational device. 

Among the comj^ liter’s greatest 
strengths arc the SPKICI) with which it 
can search its memory, undertake a cal- 
culation or solve a logical problem, the 
PATIKXCK witli which it will systemat- 
ically re^x^at questions or otherwise deal 
with refractory students, the nearly iin- 
limited capacity of its MKMOHY for 
storing all manner of information, the 
IMPAUTIAUIT with which it will deal 
with different students and changing 
characteristics within the same student, 
the consistent LOGIC with which it 
handles scqueniial information or intri- 
cate problems, and its ADAPTABILITV 
to wide variations in the manner of pres- 
entation of the same problem through 
the availability of virtually unlimited ca- 
pacity for brarxhing to alternate |x>ssi- 



bilities. In the process of performing all 
these functions, it can keep complete 
records of the entire proceedings. As we 
will see shortly, this is a singularly im- 
portant function for educational pur- 

[X>SCS. 

Among the more serious of the com- 
puter’s limitations in performing the 
kinds of educational challenges it tends 
to be given arc its rLEXIBILm’ 
being limited as it is in its responses to 
the finite repertoire which its program 
necessarily provides, its essential lack of 
ORIGINALITY, in comparision to the 
human teacher with which it is so often 
compared, its FALLIBILITY in Icnns of 
the electromechanical brcakdovsTis to 
which current computers remain prone, 
and its essential lack of PERSONALI- 
TY, or patently s^mthctic programmed 
personality, which remains a somewhat 
jarring charaetcrislic to many students 
who arc otherwise caught up in th? hu- 
manoid characteristics of the interaction, 

The process of adaptiiig the computer 
to the instructional requirements in 
medicine must include the extraction of 
mavinnim advantage from its strengths 
and careful accommodation to its limita- 
tions. 

Computers in Under fjadualc Medical 
Education 

Before discussing the computer as an 
instrument of direct instruction, I will 
briefly mention some of the ways it can 
increase the efficienc)* and capabilities 
of administrative functions which sup- 
|X>rt the overall educational system. Al- 
though they are not seen as part of the 
pur|)oses or the present paper, they are 
mentioned for completeness. The proc- 
esses of student record keeping, class 
and patient scheduling, ^ueational 
budget management and the develop- 
nent of a reference memory on a variety 
of relevant subjects are all parts of the 
computer’s potential contribution to ed- 
ucation. They have Ikcti discussed by 
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Caf/rey (4). The exceedingly important 
functions of library maintenance and 
management have been described by 
Bre.ndt (5) and will be elaborated upon 
in this conference by Dr. Nfarlin Cum- 
mings. 

The remainder of this paper will be 
devoted to those educational applica- 
tions in which the student receives in- 
struction through direct contact with the 
computer, using a remote terminal. This 
consists of a l)’pcuTiter-li!vc apparatus 
which permits both the student and the 
computer to ask and answer cjuestions in 
a fairly free exchange. There may be, in 
addition, a leIe\ision-like tu!)c for the 
display of printed information or dia- 
grams, as well as computer controlled 
slide projectors, nio\ie projectors, and 
audio recorders, for providing the stu- 
dent with sensory inputs of various 
l)pes. I will briefly examine the four 
major t)pcs of educational uses wilhin 
this selling. 

A. Information and Concepts 

Convenlion.al medical education is so 
information oriented that it is no sur- 
prise to find tli.it a fair projiortion of 
early computer b.ised instiuclinna! jiro- 
grams have been primarily information 
Oriented. A fairly typical illuslration of 
the strategy employed is represented by 
the model adapted from SjxiUVy (0) (n 
Figure 1. This mtxlel, it will W iioli('t'<h 
takes advantage of the adaptability, log- 
ic and p.it fence of the <t>mputrr to sys- 
tematically check, and if m a ss.ary cor- 
rect, the progiess of the student in ac- 
epuTing a iKKiy of iivfonnatlon. It should 
l)C clear that the computer is fully cap.i- 
blc of drilling students in the ac<pnsilion 
of information. Inforni.ition acrpiircd in 
this way, however, will generally not 
last lonf Information of the complexity 
rc<]uircd in medicine, paradoxically, is 
acx|uired best when it is no/ the primary 
center of attention, bnt is jiart of appli- 
cations that are meaningful to the stu- 



dent. As discussed earlier in the paper, 
information that i:# not alive and vivrid 
for the student is best left to books or 
some other repository. 

B. Intellectual Skills 

Essential in its relev’ancc to the gradu- 
ate physician, and hence to the student 
physician, arc the skills of identifying 
and solving problems, generating and 
testing hypotheses, and fonmdaling 
judgments. The characteristics of the 
computer make it possible to present the 
student with repetitive instances of 
these procedures for practice and mas- 
tery. It is here, it seems to me, that the 
computer holds its greatest potential in 
undergraduate nicdic.al education. Re- 
turn to I'igure I and imagine, if you 
will, a problem oriented, rather than an 
information oriented, program. The er- 
rors that the students here make are 
process rather llun confeni errors, and 
the remedial assislrmcc they arc given 
dc.ils with the coinjionents of the intel- 
lectual skills being learned, rather than 
vvilh facts being .accpiircd. b’or example, 
lluy iniglil be nlerlc'd wlu n they dr.a'V 
an unwarranted inference or nriive nl a 
judgimnt vvith insufficient evidnicc. 
And. most iui]xirtant of all. the comput- 
er’s cap.icily to rct.uii a tot<il record of 
all tr.uisac lions iu which it engages, en- 
ables the student to l>o provided with 
the cxignitive cf[uivalenl of a vidc^ ta|>e 
of his lliouglil In fact, the 

cw ipwler print out of the ]nobleni soIv- 
iTig proci ss in which th<' student has cii- 
g.igal is f.ir more flexible than a 
v icleo It is more like .a map which 
can be scanned in its entirety . :.il com- 
panal to other ni ips, for critical e valua- 
tion. Much bke the golfer who in.ikcs 
startling ]uogress through !>eing con- 
fronted vvith liis own swing, the medical 
student Ir.irning intellectual skills, when 
provided vvith the fccdl>aek of the 
print out, can make unprecedented 
strides in areas previously barely 
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touched by conventional riec.ical educa- 
tion. 

C. Technical Skiils 

The flexibility of the computer, com- 
bined with a multimedia terminal, per- 
mits the student to be presented with 
wide varieties of EKG tracings to be in- 
terpreted, slides to be read, or heart 
sounds to be deciphered, to mention 
only a few of the possibilities of instruc- 
tion in technical skills where student re- 
sponses can be immediately judged and 
repeated practice pro\id(^. An even 
more sophisticated possibility is the 
creation of complex simulation situa- 
tions, such as “Sim F,” developed by Prs. 
Stephen Abrahamson and J. D. Denson 
for the training of anesthesiologists. 

D. Evaluation 

The capacity of computers to keep a 
complete record on all its transactions 
with students permits an unusually com- 
prehensive and clal)oralc evaluation, 
l>olb of student performance and in- 
slrtictional effectiveness. The singularly 
attractive possibility whicli emerges Is 
that of compute r-iuonitorcd instniction. 
In a way wc have never l)ocn capable of 
doing in the past, student progress can 
l>c asscsscsl from day to day, rather than 
term to term, and instmctional adjiisl- 
ments made, literally, as needed. 71ic 
extent to which this can elevate our effi- 
ciency can, at present, barely l>c imag- 
ined. In addition, the memory and adap- 
tability of the compiilcr permits the de- 
velopment of a store of great ranges of 
evaluation materials, which students can 
use at any time to assess their own prog- 
ress along many dimensions of any disci- 
pline. Still further, formal accrcsblation- 
l)pc e\ ablations can l>o undertaken 
more flexibly and more reliably 'silh the 
computer. An imaginative new romput- 
cr-basc<l techniq\ic for evaluating stu- 
dents of large differences in ability, in a 
manner that exposes them to a imir[uc 
set of questions which arc maximally 



adapted to their level of performance, 
has been described by Chauncey (7). 

CONCLUSIONS 

The computer, without question, pro- 
vides the potential for an enormous 
technological leap forward in education. 
Its long range promise and consc- 
q\i cnees can barely be glimpsed, if rec- 
ognized at all. Many of the steps, how- 
ever, which must now be taken to help 
us profit most from this potential, can 
readily be seen. 

Nfost important is the need for us to 
identify, as explicitly as possible, the 
real purposes of medical school Instruc- 
tion. Wc must then define, as precisely 
ns we are able, the characteristics and 
components of the comneleneles our 
students arc to acquire. Tliesc must be 
formulated in terms that duplicate or 
parallel the eireumstances in which they 
will actually he applied following grad- 
ualion, so the practice we provide our 
students in school will l>e maximally rel- 
evant and real. Provision must also l>o 
made for effective feedback to the stu- 
dents on their s\icccss in acquiring these 
compenicncics. 

A burning question in the minds of 
many is: if vw succx^ccl in properly ex- 
ploiting the |>otenlial of the computer, 
will the live teacher l>o displaced? In an 
observation that is ( quail v relevant to 
computers, H. F. Skinner has said that, 
"Any teacher who can Ik* replaced bv a 
leaching machine, deserves to bo." The 
point is this: the comp\ilcr can, indeed, 
do certain things I'Ctter than the live 
teacher. It vv as more than a half cenUiry 
ago that Thorndike observed. ”a human 
Wing sho\ild not Ik* wasted in doing 
what forty sliccls of pajacr or two phon- 
ograph records can do. Just iK'Cause per- 
sonal leaclu’ng is precious, and can do 
what apparatus cannot, it should Ik* 
saved for its peculiar work." (8) The 
challenge Ik fore us is not how to sup- 
press computers so as to |ximit teachers 
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to go on doing those things they no 
longer need to do. The challenge is, 
rather, to exploit computers to their 
maximum and, at the same time, work to 
redefine the human teaeher s role so that 
we may take full advantage of his po- 
tential, whieh no machine of the foresee- 
able future will possibly duplieate. 

No innovation of this potential magni- 
tude is ssithoul risk. Shall we wait until 
all the data arc in and move fonvard so 
deliealely and with such caution that all 
false starts and errors will be avoided, or 
should we acknowledge the enormous 
promise now available to us and forth- 
rightly and vigorously take some 
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SUMMARY 

COMPUTERS IN UNDERGRADUATE MEDICAL EDUCATION 



Panel: Hilliard JasoA, M.D., Ed.D.^ Chairman^ Stephen Abrahamson, Ph.D., Alan Kaplan, M.D., 

Theodore PcUrson, Ph.D., Lee $. Shulmani Ph.D. 



Il is not possible to give credit in a 
summary like this to everyone’s favorite 
idea. As a matter of fact, il is difficult to 
include the most salient points of a 
three-hour discussion. However, this pa- 
per will attempt to give at least an over- 
view of the major ideas discussed. 

Our approach was first to lake a look 
at the present stale of the use of com- 
puters in undergraduate medical educa- 
tion. Dr. Theodore Peterson reported as 
to the work underway at Harvard Uni- 
versity, some generalities based on their 
work and their summary of the work of 
others. Dr. Lee Shulnian helped us look 
into the future to some new ways com- 
puters might be used in achicNing some 
objectives, particularly in the area of de- 
veloping the processes of medical in- 
quiry. Dr. Stephen Abrahamsoii helped 
us slay reality- oriented with his very apt 
and eficclive reminders of the ap- 
proaches that are necessary to evaluate 
properly what we arc doing as we do il, 
and Dr. Alan Kaplan of the Bureau of 
Health Manpower prouded perspective 
as to the hojKS, wishes and plans of the 
federal government in the area of sup- 
port of computers in medical education. 
In order to bring some coherence to a 
rather free-flowing and not always s\*s- 
lemalic discussion, eight general coiiclit- 
slons derived from Inc discussions will 
be prescptcd. In addition, Dr. Shulman's 
comments will bo presented as an ap- 
pendix to this suii^mar)'. 

•rre.ffnfrd hy Hillkrd /<jjcn, M D , Ed H. CK:\ 
fertne^. 



ITic first conclusion wdiich can be 
slated with some acron)mily, is that 
computers in undergraduate medical ed- 
ucation arc not yet in what might be 
called an implementation phase. Any 
school that does not now have a com- 
pulcT should not feel that they are some- 
now missing out on an important 
contribution to their education program. 
In fact, this leads to the second general 
conclusion^ which is, that the primary 
justification for having a computer 
would be for a few innovative and even 
unique studies of Us utility and effec- 
tiveness in medical education. 

The third conclusion which ties into 
the first two, Is that the issue of cost in 
the use of computers is a complex but a 
seemingly imj^rtant one, and there 
seemed to emerge two different jx)si- 
lions regarding cost in the wavs comput- 
ers might lx* used in tinciergraduate 
medical education. \Mien computers are 
used to do something that is already 
Ixing done, then cost w as an exceeding- 
ly important consideration. Ilic ratio of 
cost to effectiveness was one of the cen- 
tral issues in determini.ng whether what 
we now do can ^x)ssibly bo done differ- 
ently or iKlier with a computer. Rut 
there was a second ^x)sition and that is 
of the possibilities that computers bring 
that may enable us to do things we 
aren’t doing at all now and may enable 
us to reali/o instruction.il objectives lh,il 
arc not available now’. In area.s like this, 

ct the i losing plenary tession cf the fen- 
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cost becomes a far less imporlant con- 
sideration. Thai is» if we eaii use the 
computer and our studies of il to help us 
produce a physieian of higher rjualily 
than we have produced before^ then cost 
is no longer an issue. Our dedication has 
been that if we can produce something 
really worth producing, cost becomes a 
secondary issue. 

A fourth general conclusion is that the 
technology of the conipulcr seems to be 
considerably ahead of our readiness to 
take full advantage of it. The most cen- 
tra! and most pressing need that was re- 
peatedly identified was the urgency 
with which we need a far belter delinea- 
tion of our objectives in undergraduate 
medical education. These arc not the 
broad, philosophic, abstract program ob- 
jectives of trying to prCKluce good physi- 
cians; these arc the hard-to-define but 
exquisitely important instruct ion a! ob- 
jectives that commnuinicale what arc in- 
deed the specific competencies that a 
graduate vill have, if wo are to regard 
him as a worthy graduate of our school. 
These have never !>ccn defined, and as 
has been said before, if yo\i re not sure 
where you’re going you might end up 
somewhere else. T he need for precision 
in defining ohjcctives docs not require 
government sujqwrt, il docs not require 
hardware; il docs require the conscien- 
tious dedication of medical ccUicators to 
the task of working logether and coming 
to grips \nth what il is a physician is 
going to be able to do. Not only what he 
is going to be able to know, but what he 
is going to be able to do, if we have con- 
fidence In him and in the quality of our 
InstrucUom The important ;>oint is, that 
whether we go al>oul doing that expli- 
citly or iK>t, w' ) iVc done it. Without 
defining objectives explicitly, we have 
defined all sorts of objectives implicitly, 
just by doing what we do when we as- 
sign a particular class to a partiailar 
topic. Tnal i* a definition of objective. 



By failing to be explicit in objectives, we 
run a terrible risk of which we w*crc re- 
minded several limes on »he panel. That 
is, the risk of achieving some objectives 
at the expense of more imporlant ones; 
that by helping students work in a par- 
ticular area we might be failing to help 
them learn how to work in another area. 
For example, if we have students do a 
very significant portion of their inde- 
pendent learning on a computer, arc 
we perhaps running the risk of failing to 
help them with the important skills of 
learning how to learn without a comput- 
er? There arc more important questions 
than ih^t, and these relate to our cur- 
rent examina tion system. We communi- 
cate to sl\idcnls most effectively what 
our objectives arc by llic exams we de- 
sign. Another way of comnumicating ef- 
fectively our objectives is by investing a 
great deal of effort into what we put in- 
side a computer. If we convey a lot of 
ephemeral details and very short-lived 
information because that s what is eas- 
iest to put into a computer or easiest to 
examine on paper and |>cncil tests, then 
we arc communicating to them funda- 
mentally that learning is not a very rele- 
vant ac tivity, and that the act of partici- 
pating in medical education Is a game 
which you play in order to gel a degree. 
Then you end up with one of the great 
tragedies of medical education in Ameri- 
ca; the production of generations of 
physicians who arc committed to the no- 
tion that learning Is something they will 
spend the rest of their career avoiding. 
It we don’t design our programs, our to- 
tal educational programs, and our com- 
puters in a way Inal helps pliysiciaas 
dcvTlop the skills and dedication to con- 
tinue to modify themselves the rest of 
their livTS, an)lhing else vve do Is of no 
importance. 

Tlie fifth general conclunon that lies 
in very closely v\ ith the last, is that in- 
tuition just isn’t enough In educational 
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planning or computer usage in medical 
education, Intuition iias led to a pro- 
gram of medical cducatioia in this coun- 
try that can safely be described as rigid 
in the extreme. Despite the c\idcnce of 
the remarkable diversity among medical 
students, we have sonicliow intuit i\cly 
fashioned the program that tries to pre- 
tend that cverybeSy learns in the same 
Nvay, at the same speed, and is iiiUrest- 
cd in the same th ngs. All of this flies in 
the face of everything v/c know about 
people. So that ii we are going to use 
computers intelligcnily, we need to put 
aside intuition because it has failed us, 
and we have to start assembling data. 
We have to build into our computer us- 
age a \cry gcncrc'us allocation of fvmds, 
time, and effort; to start a process of 
evaluation of wliat we arc doing; and to 
desclop a rescarei base that will guide 
us in the future. A specific suggestion 
finding favtT was lire idea of encourag- 
ing and dc\ eloping regional efforts at 
research in medical ctlu cation and var- 
ious parts of medical education — cate- 
gorical sub-parts of the total picture. 

The sirf/i genrre f conclusion \s ould be 
that it is incxcuiablo to umlctlakc a 
project lexlay in the use of coinpvilcrs in 
undergraduate medical education with- 
out a major inscsinrcnt in csalualion of 
the process. 

The srt'fnf^i vwjor co/ubMioii was 
that the promise of compvilcrs for un- 
dergraduate mcdie.al education is enor- 
mous, indeed. It brings more than just 
its own utility; it is having the heuristic 
effect of forcing us to clarify our educa- 



tional objectives more precisely than wo 
ever have before, of promoting a re- 
thinking of our purposes and a re-think- 
ing of our methods that is long overdue. 
More significant still, besides being ca- 
pable of direct instruction and being ca- 
pable of direct evaluation of students, 
the computer is capable of storing its 
own processes as they arc conducted 
and becomes a remarkably effective tool 
in evaluating its own functions with our 
direction and w’ilh our design. 

The cighfli and /inol major conclusion 
is finding the proper balance between 
the live teacher and the technology rep- 
resented by the computer, which is the 
greatest challenge Inal we face. Our 
challenge is not only to use the comput- 
er inlcliigcnlly, but to rc-dcfinc imagi- 
natively the role of the live teacher be- 
cause so much of his lime and effort in 
undergraduate medical education is now 
devoted to things we already know can 
be done better by tccbnology^ If "c arc 
going to provide a quality medical edu- 
cation, if we arc going to undertake the 
remarkable com]»cx transformation of a 
college student into a professional ^si- 
lion, we have to provide him with effec- 
tive models of what it is we want him to 
become. What we need is the creation 
of live teachers who will embody, will 
not just cs])Ousc but will embody those 
things We nave decided we want our 
students to iKcomo. This a machine can 
never do and this is where the proper 
role of the live teacher begins to be de- 
fined and this is where much work re- 
mains to be done. 



INQUIRY, COMPUTERS AND MEDICAL 
EDUCATION* 

I.EE S. SHULMA.N, PH.Dt 



I Speak to yon as someone engaged in 
the study of the in a i hut in wliich people 
learn, organize s>hal lias been learned 
and employ that organized boeJv of 
kno^\^ Si rs'iee of solving 

1 . ; M'' )noeess inquity, 

I Jolni DlWi y. An i \.iniination of in- 

ijuiry can provide eduealois loiieen^cd 
with* ibe teaelihig of future nbysicians a 
source of monitorings evaluation and 

modeling that » ill serve tlieni exactly as 
\\ e have claimed the computer can serve 
llic student. We say that it is iini^rtant 
for the student to receive appropriate 
feedback so that be has a constant per- 
ception of how closely be is approximat- 
ing sonic ideal model of bow a ph)'sieian 
operates. If wc as educators arc going to 
presume to program the computer to 
provide that kiiiA of stivicc for the stu- 
dents wc need \cry clear ideas in our 
ouii minds of what arc the criterion 
models of medical practice's medical di* 
agnosiSs interview ing, bi^toiy taking and 
physical examination Hic^e arc the 
models which the computer must em- 
ploy, Wc ought to specify these medds 
(>cforc we design any jirograni, I do not 
know that w c hav e accomplished this, as 
yet. 

TyjKS of Menhh 

I low* do wc generate this kind of 
niodcP One approach, which wc might 
call the experience ruodfh is tc make 
careful observations of men and ’a omen 
who are identified l>y the cr>rnmunity of 



.medical practitioners and scholars as 
being critcnol physicians, fhese indixid- 
uals represent in their behavior the fin- 
est kind of medical practice wc can 
identify. Wc then attempt to abstract 
from a careful analysis of what they do 
under variou\ conclitionSs and finally 
employ tliat ^abstraction as the model wc 
pul into our computer so that it, in turn, 
can now’ react and give feedback to the 
medical student who is learning. This is 
the sort of thing that is done in the heu- 
ristic programming of a computer tvi 
play chess or prove mathematical theo- 
rems. Wc do ml have in the medical lit- 
erature extensive studies of the actual 
bchax*ior of fine diagnosticians in suffi- 
cient detail to lead us to a me del such as 
this one. This kind of research w ill l>o 
necessary' in order to make medical in- 
struction significantly more effective. 

TTif second approach is to develop 
fogiVtjf tnakU. Wc ask ourselves, not 
bow do good practitioners praciice or 
good diagnosticians diagnose, but rallier 
what apjKars logically to l)c the model 
for inmhrv, be it medical inquiry or any 
other Kind of inquiry. The illuslralivc 
model I shall now* descTilx', in simplified 
fonn, is derived from Dewey’s philoso- 
phy of inquiry. It has directed my re- 
search efforts over the last six years. 

A yfoifcl of Inquiry 

Wc l>cgin with the stage of problem 
jrnsfng. Nfosl real inqtiity docs not be- 
gin the wav wc represeid it in schools. 
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Thcrc» we present students with prob- 
lems to solve which ha\'e already been 
very carefully formulated. Ratlier, rmicli 
of inquiry in a natural setting begins 
with the problem solver or inquirer 
being placed in a situation that is terri- 
bly ambiguous and undefined — where 
the edges are blurred and the distinc- 
tions are not clearly niade. His first task 
is to sense where the problems lie and to 
distinguish what is problematic fron'j 
what is not. NVe all iccognirc that two 
individuals can encounter essentially the 
san-ic situation, which one of them will 
see as problematic while the other will 
merely nod and go on. Problem sensing 
is a ver)' subtle kind of process. We do 
lainfully little to train people in prob- 
em sensitivity. It may be, however, 
differences in the ability to sense prob- 
lems that distinguishes the medical thor- 
oughbreds from the horses that pull the 
milk wagons. 

Once a problem has Ikcji sensed, it 
must l>e put into words or categories. 
That is, it must l)c form\i!atcd. John 
Dewey observed that a problem well 
fornuilaled is half solved- The manner in 
which wc formulate any sensed problem 
delimits the kind of resolution that wc 
are ca]^ablc of generating. A problem 
formulation determines the kinds of in- 
fomiation wc are going to seek out in 
our search, the kinds of hyjxrlhescs wc 
will entertain or reject, and the kind of 
solution that we will admit as |X>ssiblo. 
Froblcm for mutation is the second stage 
or aspect of inquir)*. 

The third 1$ search. Search can bo 
both external and internal. Wc not only 
corKlnct search by asking questions of 
patients, ^xrforming a physical examina- 
tion, calling for certain kinds of lalxira- 
tory tests and generally gathering infor- 
mation from the ovjtsid^. There is also 
internal search. Wc have enormous 
storehouses of information Ixlwcrn our 
cars and, |x>tcntially, in ovrr computers. 



The question is how much of this infor- 
mation do wc seek, and which of it do 
we seek? The interiictioii of the internal 
and c.xtcrnal informatioii-gathcring con- 
stitutes this total process of searching, 
The search is defined as the sequence cf 
operations, ejucstions, movements, frus- 
trations, revisions of tactics, and the like, 
that tlie indiWchral undertakes in order 
to transform the problcm-as-formulatcd 
into a ]xrsorally-fclt resolution. 

Problem resolution occurs when we 
decide that wc arc finished and the 
problem which initially keyed off the in- 
quiry is solved. In the studies we have 
conducted of inqiiiry in teachers, we 
find cnoMnous variability anion g indi- 
viduals in what they will acce pt as r. res- 
olution of a problem. Ihis is closely re- 
lated to their formulation, Imt it i< aho 
related to individual differences in pvr- 
sonality, values, intelligence and train- 
ing. rnforlunafcly, differences atfrilmt- 
able to training arc \isually accidental, 
because rarely do wc have a clear model 
in iniml when wc engage in instruction. 

It is very (mjx>rtanl to note at thi< 
point that the model of inqviiry dc- 
scrilKcl above does not require a rigid 
sequential ordering of steps wherein all 
inquiries can W sulxlixidod into fotir 
simple stages representing each of the 
four al)Ovc-namcd processes. '^The natti- 
ral process of inq\iiry ha.s these four 
parts as somewhat imlc|X'ndenl coni|>o- 
nents. Any particxilar inquiry will look 
very much li<e a computer program 
wilK its m.my loops and ciigrr ssions. The 
steps of inq\iiry act as the basic opera- 
tions in the program. Thus, U would not 
l>o unlikely for a given subject to sense 
tbal a particnilar problem exists, proceed 
to fornvilato it in a particular way, initi- 
ate search activity in order to gather 
data concerning the piobbin. rcalirc as 
a result of the initial search that his 
]iroblcm fornmlation is irislxcottcn, rc- 
tvnn ar>d re formulate the problem, while 
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doing so sense an additional problem or 
t^^’o, thus reformulate once again, and so 
on. 

Medical Inquiry 

NVhat happens when you take this 
kind of model and attempt to apply it in 
the context of the medical situation? 
The diagram of Figure 1 suggests that 
the input of a specific ease interacts 
with tne organized body of knowledge 
possessed by the physician, his long and 
short-term expectancies of what he will 
encounter and the general strategies or 
heuristics he has learned, to generate as 
output the rnedieal diagnosis, treatment 
and evaluations. The full complexity of 
medical inquiry is surely not represent- 
ed in this diagram. The relations be- 
tween this model of medical inquiry and 
the general inquiry' model arc only sug- 
gested here, A far more detailed analysis 
must await much additional effort. 

I think it is fair to say that the acqui- 
sition of organized bodies of knONvlcdge 
has been the major focus of discussions 
in this conference, with the assumption 
that the computer is useful in making 
more efficient and effective the manner 
in which these bodies of knowledge arc 
acquired and organized. Not yet an- 
swered satisfactorily is: How do alterna- 
tive ways of organizing a body of 
knowicilge have different consequences 
for directing the process of inquirv? 
This is the name of the game in meal- 
cine, as it is in teaching and in other 
complex problem solving tasks, \Vc 
tcaeh the organized bodies of knowl- 
edge, not to create little living libraries 
of our subject, but rather to make these 
bodies of knowledge maximally retriev- 
able and useful in Inc service of inquiry, 
fs Socratic leaching, whether live or by 
computer, the most effective way of ac- 
complishing this? Is drill to the point of 
ovxrlcaming the most effective w*ay? 
The most general question for the differ- 
ent aspects of the inquiry process we are 



tr)ing to teach in medicine is: Which 
approaches to learning arc most likely to 
assist us in achieving our multiple goals? 

The computer must be introduced 
into the flow of medical instruction as 
part of a systematic strategy of educa- 
tional development. First a map of ob- 
jectives must be drawn in very' specific 
and o^Kralional terms. Those will include 
organized bodies of knowledge, certain 
kinds of expectancies on the part of the 
physician, and strategies of inquiry'. All 
these affect the conduct of medical in- 
quiry, The future impact of the comout- 
cr on the practice of medicine is litcly 
to be reflected in a shift of instructional 
emphasis away from the organized bod- 
ies of knowiedge to the strategies of 
knowledge- retrieval and inquiry. Next, 
the materials representing possible paths 
to the terminal objectives must l>c devel- 
oped. Third, the materials and objec- 
tives must then dictate the combination 
of software and hardware most ap- 
propriate to guiding our students in 
traversing the map. Fourth, the total in- 
structional system so designed must \)C 
implcmcnteo \iiidcr carcfinly monitored 
conditions. Finally, the results of that 
implementation must be evaluated in 
terms of the original map of objectives, 
This evaluation could lead to changed 
instructional tactics, modifications in the 
map, or both. If we too quickly try to 
nK>ve into the implementation stages 
without prior, or at least simullaneovis. 
effort in the planning and design stages, 
we arc likely to generate more chaos 
than progress. 

nefeorc/i in Medical Education 

1 shall conclude with an urgent plea I 
make not only to medical educators but 
to educators of all kinds. For too long 
vve have allowed complex and exjKnsive 
educational operations to be piloted by 
the seats of our pants. We dedde what 
we do on the basis of our reactions to 
what was done to us. This can work in 
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two \va)’s. We can either repeat our own 
cxi>eriences or react against them and 
do the opposite. Both arc equally unten- 
able. 

\Vc must get beyond this level of 
doing things just because we intuitively 
fool this is the best way to do it. We 
must attempt to study carefully the 
processes in which wc arc engaged. I 
am not suggesting that people in medi- 
c.i! education nin to the studies of B. F. 
Skinner and try to extract his findings 
and apply them directly to medical edu- 
cation. That \N‘ill not succeed^ because 
medical education is not the same as 
teaching pigeons to play ping pong or 
even 6-ycar-olds to count. Rather, we 
need to create a discipline of the psy- 
chology of medical education that will 
bear a relationship to the general psy- 
cholog)* of education and learning — a 
relationship that is parallel to the rela- 
tion medicine as a discipline bears to the 
basic sciences. Clinical medicine is an 
intermediate discipline between the bas- 
ic sciences and direct medical applica- 
tion. Tlic applied discipline of the psy- 
chology of medical education must hasc 
analogous characteristics. It does not 
now exist except as a set of intuitively- 
derived dogmatic statements. How can 
the creation of this field lx* accelerated? 

The scojx' of systcnMtic research 
which is needed in medical cducatton is 
great. In order to achieve that sco[X* as 
quickly as |>ossible, wc must irnpicment 
broad programs of research in nodical 
teaching and learning, making r>esl pos- 
sible tjsc of the small numlxr of current- 
ly ac“tive personnel in that field. 

A strategy I would recommend for 
yovir ct)nsidc ration Is the following: 
us establish a scries t>f .'/fdico/ Edwro 
tion Rcseurch I^boraiorics around the 
country. They could l>c organized on ei- 
ther a regional basis, or through joining 



together units which sha^e interests but 
may be in different parts of the country. 
Each laboratory would be dedicated to 
a specific research mission in medical 
education and would draw upon the re- 
sources, facilities and research popula- 
tions of all its cooperating units — uni- 
versities and hospitals. Such missions 
might include the specification and defi- 
nition of objectives for medical educa- 
tion, the development of evaluation 
models and evaluation instruments in 
medical education, the uses of technolo- 
gy including the computer, the study of 
the medical inquir)* process and studies 
of the acquisition and use of organized 
bodies of medical knowledge. 

Hie Medical Education Research 
laboratories must also assume a train- 
ing function in order to produce the re- 
search and development jX'rsonncl need- 
ed to disseminate, imidcrncnt and evalu- 
ate the research-based approaches 
which emerge from the laboratories. 
This medical education research effort 
cannot be left to individual medical 
schools, but must he coordinated sys- 
tematically on a nationwide basis. 

Ivan Pavlov is said to have told his 
students, ‘‘Ideas and theories arc like the 
wings of birds — they allow man to 
climb and soar to tlic heavens. But facts 
arc like the 'Atmosphere against which 
those wings must l>cat, and without 
which the soaring bird will surely plum- 
met back to earth. ** \N'c arc at a moment 
in the history of medical education 
when wc must buttress otir m)Tiadio 
ideas and theories aloul the best forms 
of medical education with painstakingly 
collected facts. In this effort the com- 
puter will l>c an indispcnsiblc tool. 
Without this research effort, the eom- 
piter in medical education could be- 
come our newest enemy. 
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COMPUTERS IN CLINICAL MEDICAL EDUCATION 



DON'AI.I) A. B. LINDBKRC, M I).* 



There are a number of sources 
from which one learns clinical medi- 
cine. Computer s)’stenis have poten- 
tial roles to play in providing access 
for the student or physician to infor- 
mation. We will examine examples of 
sueh systems. 

The student or physician learns 
about medicine from four major 
sources. These are: common sense 
and lay beliefs; by examining the 
facts of patient care; by ex|x)surc to 
purely didaetic infonnation; and from 
the ^x^rsonal guidance and precepts 
of his teachers. 

Facts of Patient Care 

One Icams mcdieinc from the op- 
portunity to jKrsonally care for pa- 
tients or to see care given. This is a 
valuable, but limited experience. One 
cannot see examples of each and cv- 
rr\' disease, treatment, outcome, and 
evaluation of sueh processes. Consc- 
nuently one learns by drawing U|>ou 
tne experience of a whole institution. 
A computer s)*stem can provide valu- 
able assistance merclv by serving to 
retrieve patient rccorc^s, to search the 
hospital files for records, and by |Kr- 
milting comparisons to bo made Ix’- 
tvveen different groups of patients or 
different trctitments or management 
schemes. ( 1 ) (2) 

In such cases the re«al learning is 
done by the student reading and 
studying the usual vsTiUen hospital 



record. The computer has merely 
guided him to the correct record. 

Inquiries into the records of labora- 
tory determinations can also be a 
source of learning. Systems have been 
provided so that the inquirer can sec 
the frequency distrihulion of results 
of a determination during past hospi- 
tal experience, and so place his pa- 
tient’s results in a pcrsjx'ctivc of the 
institutional experience (3). Tlic sys- 
tem provides the ability to retrieve 
any item of laborator)’ studies for any 
patient, or alternatively to retrieve 
and display categories of results 
(such as. those patients with cholcs- 
lerols greater than 300 mg^‘ or bv 
comjvounding categories (such as by 
also rtHpuring that the patients be of 
a particular race, or sex, or that the 
results have occurrcxl during a certain 
time period). It is also possible to rc- 
rpiire in snob an inquiry that the re- 
sults of a second or of a third deter- 
mination also meet certain specifica- 
tions. Tlic net result of sueh com* 
]>oundcd inquiries is merely the dis- 
play of data or the listing of patient 
identification numbers — nothing 
creative or innovative. Yet such a fa- 
cility permits the student or physician 
to asV questions of the mass of \x)len- 
tial ''teaching dat.a" which might oth- 
erwise In' difficult to use via manual 
record room systems. 

An additional Ixncfit of the facility 



'f>tffr-iOf, Str^kat CerJer Conprifer Prof^ren, t^^rnVrni7y of ytc<tknt Center 



to retrieve the results of laboratory 
determinations is prousiors to the lab- 
oratory director of added (juality con- 
trol capability. lie can, for example, 
specify \ ery closely the format of re- 
ports, the rec[uircmcnts for release of 
a report (I. e. tolerable “liniits” and 
special rules), the permissible preci- 
sion for reporting, the extent to which 
cpiality control sera for a given batch 
run may vt^ry from the most precise 
estimate, thi’ extent to which aqueous 
standards may vary from their 
expected curvilinear relationship. The 
laljoralory director can take into ac- 
count the relationships between the 
results of multiple simultaneous de- 
terminations in formatting his “limits’* 
sptems (4). A simple example of this 
kind of rule is the requirement that 
hemoglobin concentration and hema- 
tocrit bear a sensible relationship to 
one another when both estimates arc 
available for the same blood sample. 
Computer systems for examination of 
laboratory’ clctenninations can permit 
the laboratorian to ask such questions 
before erroneous reports crexp into 
the patient care system. Ulien rcjx)rts 
deviate from the “limits" systems, the 
resident physician lias an opjwrtiinity 
to re-examine the situation and ap- 
prove or disapprove the reix^rt. Such 
constant re-cxarnination of premises 
and performance, as well as the crea- 
tion of new limits and rules selves an 
educational and training fuiKtion for 
the house officxr and the K^ilnirator)’ 
director as v'dl. At the vexy least, 
such s)*stcms direct ones attention to 
the exceptional clinical ease, ideally 
the case requiring profcssioral inter- 
vention and judgement. 

Didnciic /n/t>rfJk3rion 
The published medical a?ul sden- 
tific literature is indexed by the Ka- 
lioml Idbrajy of Ntodic^ne, ]niblivh- 
ers of the Index Mcdiais. This cita- 



tion data base itself can be searched 
by the NLNf computer system, and 
hopefully this system will be replicat- 
ed at many other medical centers 
(5). 

There is noNv the potential to store 
considerable quantities of information 
on microfiche systems, which in turn 
can be indexed by ccniputcrs (6). A( 
present one cannot ( practically 
speaking) transmit the images of mi- 
crofilmed text pages over long dis- 
tances. Perhaps image transmission 
(c. g. "Picture Phone") will in the fu- 
ture make a reality of the dream of 
browsing through the library via the 
computer. 

An experimintal program at Mis- 
souri (called Consider) attempts to 
allow the student or physician to scan 
across a broad sj>ccirum of medical 
knowledge. The data base employed 
is a magnetic tape version of the 
AMA Curnnt Midical Terminology 
(CMT), kindly provided by Or. 13\ir- 
gess Gordon. 0\ir programs |xrniil 
the inquirer to l)pc onto a computer 
tenninal a statement i f the signs, 
s)inploms, or other findings froii a 
patient. The system then seartius the 
formatted statements of CMT and 
displays for the docior a list of the 
names of diseases in which the find* 
ings occur in the eoml inatioiis speci- 
fiw, and hence which mvisl l>c ^n- 
sidcred. Medical students have found 
the program challenging and often 
ediicaticmal. It is amazing how Jong a 
differential diagnosis one can eucou li- 
ter and how often the teacher has for- 
gotten much of what is contained in 
Dr. Cordon's slim volume. 

Recorded audio medical Ueturcs 
have been dcvclopcel at the Universi- 
ty of Wisconsin (7). Under such a 
s)stcm, the user c.tn telephone to the 
message center, retjucst a partiailar 
lecture, and ge; the benefit of a care- 
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fill presentation without leaving his 
office. The original audio tapes have 
been loaned by Wisconsin to various 
Regional Medical Progranis and other 
educational groups. At Missouri the 
Wisconsin tapes are utilized through 
an automatic answering apparatus so 
that the operation is totally unattend- 
ed. The critical ingredient in such a 
system clearly is the necessary high 
quality of the lecture material. Com- 
puters enter the picture only on the 
technical side. Switching many in- 
coming calls between hundreds (or 
potentially thousands) of recorded 
lectures will bo a computer job, Simi- 
lari}', proiiding svstems so that the 
medical user may learn of the exist- 
ence of an appropriate recorded lec- 
ture may well Ic a future ■ompuler 
job. 

Research and Anafyus 
Computers have of course !>een uti- 
lized for the statistical and/or imthe- 
malical analysis of research data for 
many years (8). Recently computer 
systems have made it iKissiblc to as- 
sist in learning clinical medicine by 
re -examining certain medical tcch- 
nirjncs. For example, uw of Slack's 
automated patient history taking sys* 
teni (9) |xrmits the doctor for the 
first time to rc|HMt the process of ob- 
taining a patient history. 1hc same 
can be slid of the inlcrprelaliou of 
electrocardiograms by computer sys- 
terns (iO). ()nc probably cannot rr- 
learn a faci or concept in <piitc the 
s.aine way (with or without a comput- 
er). Even so the advent of "Computer 
Aided Instruction" systems should 
jx^rmil a far more careful study of 
learning processes in those who learn 
through computer terminals than has 
ever heen possilde in the pa^t. 



It is beyond the scope of this paper 
to review the numerous excellent ven- 
tures into true “computer diagnosis". 
1 refer to the work of Lusted and 
Ixxlley, Warner, Ix>dwick, Reichert z, 
and Moore and others. Iri all eases a 
<x)mputer system is requiring certain 
observations or measurements from 
the physicians before attempting to 
calculate a diagnostic likelihood. The 
effect on the user is to demand great- 
er precision and C'Oiisislency than usu- 
al, Often the program guides him by 
asking "the next logical cpicstion”. 
For example before the lyodwick and 
Rcicherlz program will yield up an 
opinion concerning the likelihood of 
bone tumor, it requires (he resident 
or fellow to make seventy observa- 
tions and/or measurements on the 
rex^nigenograms or patient. F^ven af- 
ter the ui agnostic programs have 
evolved to the mininniiu nunilxT of 
true cliscTiminating factors, it seems 
likely they will still !>e able to ask for 
observations and meauiremenls 
which will help the l>eg Inner learn 
medicine and the pr.utitioncr n mein- 
Iht it. 

There arc inniinu rahle i xamnlrs of 
computer terminals and apparalmcs 
designed to give the resv archer a new 
w av to visualize his data. Mulli-dimen- 
sional plotting and display, symlwlie 
representation of data, conveTsitionil 
incxlc of statislleil atialysis - all offer 
real potential for insightful analysis 
of medical data from groups of pa- 
tients. It remains for us to learn to 
use modern display devices so as to 
"give a new dimension" fo the data 
for the individual patiu t at the time 
of his illness. 
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SUMMARY 

COMPUTERS IN CLINICAL MEDICAL EDUCATION 



Panel: Donald A. B. IJndbtrg, M.D., Chairman, Robert Cremes, M D,, Nlitchell Schorow, j^hn 

A. Starkweather, Ph D,, James Sweeney, Ph D. 



Dr. Robert Greenes described llic 
fresenl computer system at the Massa- 
chusetts General Hospital which utilizes 
a conversational mode for the collection 
of information for the patient medical 
record. This system uses a branching re- 
sponse approach to kcyl>oard entries on 
the part of the tnedical user. Both tclc- 
l)*pe terminals and cathode ray tube ter- 
minals arc employed. The latter include 
a sj^cial dc\*icc produced at the Massa- 
chusetts General Hospital that consists 
of a number of fine parallel wires cross- 
ing the face of the cathode ray lube 
screen. NVlicn the medical user touches 
one of the wires to indicate his choice of 
a response displayed on the computer 
tennina), his Inxly acts as capacitance to 
activate a circuit whereby the computer 
senses his choice. 

Dr. Greenes stressed l>olh the need 
for relatively carefully formated entries 
in the patient record and the need for 
the medical user to enter free fonu re- 
sponses on the computer terminal. In 
some eases the standardi/ed or formated 
resjx)nscs are ‘fiteci fklcP although this 
is not always necessary, rroerssing of 
the totally free form entry is jxissihlo be- 
cause the conversational mode has al- 
ready contributed a consiihrablc struc- 
ture to ibc data. 

This is being uliliyid in an ex- 

|>erimci>tal mode on a regular basis for 
three patient record data collection com- 
pilations. These are the automated pa- 

*ncscrjcd hy fX rv:W .\ R LiruU cr^ .\f. /) , j 



tient history, the conversational mode 
system whereby the M. D. can answer 
the results of his physical examination of 
the patient, and a system through which 
the radiologist is able to enter his 
impressions following interpretation of 
roentgenograms. 

Dr. Greenes, Dr. G. Goto Bamett, and 
their associates want to extend the cx- 
pcrimeotal system in order to investigate 
new" applications and new" areas in the 
hospital setting. Dr, Greenes stressed 
that one of the advantages of this ty’pc 
of system, especially in the example of 
the radiolog)’ system, is the ability the 
system gives medical users to monitor 
Inc thoroughness with which x-rays arc 
c.va mined. This can play a definite role 
in the educational jiroccss of residents 
and young physicians stud)ing racliolo- 

Dr. Greenes did not see any major 
prohlein in the e.xisting hardware or 
computer soft ware. major problem 
at the monitnl is the extreme amount of 
time inxolvttl for the radiologist to 
produce a dictionary, branching code, 
instructions and the thesaurus capability 
requircti by such a data collection sys- 
tem. il is |XDSsible that a radiologist 
could siKud the l>cller part of a full year 
in designing a snlem to handle only a 
|>oilion (jxrhaps as small as fifteen jXT 
cent) of the total dictionary and asso- 
ciated systcuas rcfjuirrd foi' the field of 
diagnostic radiology. Consequently, the 

nil CX:i>t’v:n. ct the jfy r/ 



hi 



56 



limiting factor in this clinical data col- 
lection system is the amount of time 
possible for clinieal physieians to con- 
tribute to building new jx>rtions of the 
system. 

Dr. James Sweeney described the his- 
tory of the Tulane information process- 
ing system as well as the present more 
advanced version known as GIPSY, lie 
stressed the general problem of estimat- 
ing the true iidoimation content of data 
collected tlirough computer systems. Dr. 
Sweeney’s system deals only with infor- 
mation which ran be represented by al- 
phanumeric characters, and only infor- 
mation as it is perceived by the human 
"sensor.” The major objective for indi- 
\adiials designing computer systems is 
not so much that the system need \yc a 
total and j>erfcct reflection of the real 
data, but that it not lose any iiifomialion 
which has been perceived by the human 
obsersxT. The syste m then iKcomes a i ?- 
posilory for the observations of the hu- 
man being, and should prosadc him ca- 
pability to inf|uirc into this reposilor)’. 
In orclcr for the systems to be at all 
manageable and realistic in a medical 
selling. Dr. Sweeney felt it essential to 
strip off from the total information of 
the hospital all lh.it is not likely to have 
essential worth in ]iaticnt treatment and 
medical teaching. Dr, Sweenev dis- 
dained the two extreme \icws: the vie w 
that all clinical information is exceeding- 
ly unreliable and "noisy" and the \icw 
that leads some people to wish to er»llect 
all information connected with patient 
care. Dr. Sweeney characteri/cd the hit- 
ler approach as the "pack rat principle" 
and |K>inted out that one sho\i1d have a 
fairly good idea of the use to which in- 
formation svill be p\il before committing 
oneself to a comp\jter data collection 
sy.slrm. 

Dr. Sweeney stressed that the medical 
record has very many functions. It is 
not* in his opinion, well organi^(d at 



present for purposes of scientific in- 
quiry. He also raised the question 
whether medical people bcccnic well 
trained m the normal course of their ed- 
ucation for conducting scientific inquir- 
ies into information systems like the 
present paper medical record, Kven 
with computer rcjX)silorics of mcxlical 
information, Dr. Sweeney ff'U that some 
nesv approaches and |>ossibly some for- 
mal education of medical personnel 
might create a more useful total system. 

At present, Dr. Swcciicy is supcri^is- 
ing the reprogramming of his ge neral in- 
formation processing s)^tcm to comply 
svith the needs of I.B.Nf, 360 computing 
hardware. He expec ts to be done within 
a year and cxpccis the system to be gen- 
erally usable by other investigators, both 
medical and general scientific. Dr. 
SwTcney’s advance planning includes a 
scheme to proside on-line in(juiry to in- 
vestigators into their research data files, 
using computer terminals. His mode for 
collcelion will remain for the moment 
the pajKr form niili/ing certain "self cn- 
roding" principles developed oxer the 
years by him and bis group. 

Dr. Sweenev aclnowlctlgcd the cxisl- 
cnec of a problem in the existing mann- 
factvircr-svipporlcd programming laii- 
giMges, On the other band, be saw the 
major problems to bo still in the area of 
definition by mceb'eal jxrsonncl of the 
uses to Inch information will Ix^ pul. 
careful systems design, and the jxvssibili- 
Iv for innovative vises of exxnputcrs to l>e 
t1csTlo|xd in the future. Ho noted that 
the inquir)* cap«ibilily be plans lo devel- 
op has the possibility to gise a new di- 
mension lo the interphase lx?t\vcen the 
human and Ihc conipvilcr data base. He 
bad devclotx>d interactive inquirs' capa- 
bility into nis s)’stcra using second gen- 
eration hsrdware. Under these cvmeb- 
tions. the rcsjxmsc time in the ss^tem 
w.vs three to four minutes. He feels that 
his new sssteni can reeliKO this to tlir .v 
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to four seconds, and that response times 
in the range of one second will probably 
be required in order to obtain optimal 
interaction between a research inv^esH- 
galor and his data base. 

Miichcll Schorow described the work 
that Dr George Miller and he arc doing 
in primarily addressing themselves to 
the problem of educating medical per- 
sonnel for clinical problem solving. The 
systems with which tlicy are working 
permit them to create within the com- 
utcr system the patient history data 
ase utilizing a single acinal patient his- 
tory, a combination of actual patient his- 
tories, and where necessary, artificial 
constructs. The computer system per- 
mits the student to "Ircaf’ the patient by 
permitting him to ask the system for in- 
formation .about the patient and his re- 
sponses utilizing virtually free form nat- 
u.'al language. Tor example, the system 
presents to the stvident the information 
(hat he has a “patient who is a forty- 
nine year old white male with low back 
pain of two yc\ars duration, of increasing 
scNcrity for the past six months,"* Ibc 
student is then given thc' opj>ortmiily to 
make diagnostic investigations and 
treatment of (he computer **paticnt.** 
Thc rcsjvonscs of thc system arc de- 
signed to cx)ntributc to the student’s 
gaining clinical aaunen and good jtidg- 
nicnt 

Mr. Schorow and Dr. Miller arc utiliz- 
ing l.n.M. 1.500 computer hardware and 
writing in Courscvvritrr H language. At 
present, they Ixlicvc that their system is 
capable of giving totally appropri.atc rc- 
sjwnses to the student’s free form ques- 
tions in alx)ul seventy |>er cent of (he 
c^scs. This Is in contrast to their l>clicf 
(hat a minimum of ninety per cent ap- 
propriate rcs|>onscs arc necessary in the 
long run for the system to Ik a complete 
success. In fact, (heir system is still in an 
experimental stage, although operating 
in a gratif)ing manner. 



Another major obstacle for these in- 
vestigators to overcome is the amount of 
author time required in order to create a 
good student teaching program. At pres- 
ent, about forty hours of author time are 
required in order to produce one to two 
hours of student teaching material. 
Nfembers of thc panel were quick to 
note that ev’cn the forty hours represents 
a very considerable improvement over 
the amount of author time which was 
required by other computer-aided in- 
struction systems. In many cases in thc 
past, one hundred to two hundred hours 
on the part of the author have been 
needed in other systems to create one 
hoiu' of student teaching material. 

Response time of thc present Schorow 
system averages from four to five sec- 
onds, vv^th a maximxnn response time of 
thirteen seconds, Mr. Schorow fell that 
any resjx>nsc In excess of eight seconds 
in such a system was totally unsatisfac- 
tory, since the delay left the student in- 
tolerably impatient. His feeling is that 
within thc next two years, the res^Xinsc 
times of (heir sptciu will have been re- 
duced to acceptable levels, 

Mr. Seborovv stressed the need to 
have built-in objective evaluation criter- 
ia in the development of all such sys- 
tems. I!c did not feel that bis own sys- 
tem is sufficiently far advanced at pres- 
ent to report the complete evaluation at 
this liinc. His nncctlotal experience with 
llic system encouraged him to l>clicvc 
that the system was (piilc acetp table, in 
fact, exciting lo medical student users. 
A1 k)uI one and one quarter hours arc rc- 
qviircd for each sliidctU to complete a 
single clinical mcelfcal problem. 

Tlic problems presented to Mr. Scho- 
row in creating a system capable of ac- 
cepting totally free entries on thc part of 
thc student were described as serious 
Oi'.cs. Ill erdev to accomplish (his, (he au- 
thors have utilized prcjKsitional and 
projx>si(ional calculus, and other ad- 



50 



58 



vanccd theoretical information handling 
technics within the programming sys- 
tem. He seemed to feel that these tech- 
nics \\ill ultimately prove adequate to 
the problem. 

Dr. John A. St arV weather described 
bis professional obligations at the Uni- 
versity of California. These include the 
duty to improve education for both the 
medical student and medical profession- 
al community as well as nursing stu- 
dents and person neb cnel other medical 
and paramctlica! groups within tfic en- 
tire medical tenter complex, In order to 
accomplish the task, ho dcscribcrl the di- 
\ision of his central coniputing staff into 
groups corcomed with education, re- 
search, and patient care, and operating 
as separate units. All three of these 
groups as well as the scientific users, 
ntili/c a single computer facility* in the 
foiTU of an MhM. 360/50. As a conse- 
quenie, Dr. Starkweather ulili/cs 
OS/360, operating in multiple fi\c<! par- 
liiions, including tele- processing parti- 
tions as well as batch -processing parti- 
tions. A second smaller computer 
(3G0/20) is included as part of the s)s- 
tenu The interphase between the two 
systems Wing '*HASr * input /output 
spooling routine. 

Dr. Slarkwcalher stressed tint it was 
jKJSsihle to serve this diversity of com- 
puter users with existing IccluuVs. On 
the other hand, he emphasircsl his feel- 
ing that ideal and optimal application of 
a compuU r in metlicinc, espt'cially com- 
puter-aided instruction systems, <!eli- 
nilrly rcejuircs new programming lan- 
guages. ihesr languages must inter- 
phase between the medical educator 
am! the various rigorovrs and complex 
requirements of OS/360 and its Jot> 
(Tontrol Language. One such attempt in- 
volving Dr. Starkweather's new pro- 
gra mining language system PILOT w,\s 
dcscrilxHl. PI LOT is itself written in 
PLt programming language. The intent 



of this system is to aid in leaching prob- 
lem solving to students and in teaching 
the method of inquiry into patient prob- 
lems to medical personnel. The ultimate 
aid of the PILOT system is to permit 
the medical educator to create his leach- 
ing routines with a minimal amount of 
programming knowledge, and to permit 
him to do some file handling through re- 
mote computer terminals when the sys- 
tem is finally completed. Dr, Stark- 
weather stressed his belief that the pres- 
ent tools for inquiring into computer 
systems, for teaching of problem solving, 
and for simulation as a means of com- 

f nitcr- aided instruction are si ill cxlrrmc- 
y primitive, 

Lven so, computer systems have the 
potential for giving appropriate rc- 
sjX)nses to relatively free inquiries on 
Inc pail of students. Dr. Starkvs’cathrr 
felt that at this very mom<‘nl, the basic 
approach of branching coinvrsMional 
mfxlc structured iiKjviiry presented by 
I>, Hobcrl Greenes from the Massachu- 
setts General Hospital sjstom probably 
is closer to being an ojKralional reality 
than the free form ciUty^ sj’stcms. In es- 
sence, lie behoved that tbc matrix of 
<jue.slions and responses used al Nfassn- 
ch VI sells General Hospital ts presently 
totally i call Stic, but that the system he 
himself is striving to create for handliiig 
fri'C form inquiries offers a greater po- 
tential in the long run, and that this jx)- 
Icnlial can nhiinalelv be achicveib 
l>r. Starkweather stressed two addi- 
tional conclusions: cue, that the facility 
of a large computer to store vast 
amounts of information, whether patient 
care information or didactic information, 
will ill fact make a major contribution to 
medical leeching; two, the extreme irn- 
|x>rtancc in the long run of ihc compul- 
rr's ability to provide tlm knowledge of 
inauv nuxlical authorities to help one 
single student. 

Dr, Starkweather presented in some 






I 

I 

I 




detail an application of his program- 
ming system in which a student can 
learn to conduet a psychiatric interview. 
A characteristic of this system is the 
ease witl* which the novice computer 
user can answer the system and begin 
his learning experience. Dr. Starkweath- 
er stressed the fact that although the 
system is obviously a simulation experi- 
cnce» the individuals using it seem to en- 
ter very seriously intc the simulation. 
For example, individual users seem to 
be compelled to quickly answer some 
sort of response to the computer termi- 
nal questions, even though they would 
prefer to think over the answer and even 
though the question may be an awk- 
ward one. In this respect, they seem to 
forget that the computer tcmiinal itself 
is so utterly patient. 

At present. Dr. Starkweathers major 
programming system is operating well 
within the incdical center and has good 
language handling capability, although 
it has some temporary liniilalions on Inc 
nuinlKr of terminal units it can serve. 
The language is designed to aid the user 
hy offering conversational motlc assist- 
ance to him while his programming ef- 
forts arc l>eing condiiclcil. The system is 
based on a former similar system with 
which Dr. Starkvx eathcr had had cxqKrl- 
enco, namely Coinput est on the l.D.M. 
1620. 

The major problems Dr. Starkweather 
dcsciibcd involve very serious limita- 
tions in present manufacturer-supj>orlcd 
piogriimining lar^guage. He plans to cor- 
rect this proljlcm by extending his own 
compiler/fnlerprclcr programming sys> 
tcin. He, the other ineml>crs of the pan- 
el, and numerovis members of the audi- 
ence were in agreement concerning this 
issue: they all stressed their desire for 
federal government and marrtrfactnrer 
sup|x>rt lor the dcvTlopmcnt of new lan- 
guages for mcelieinc and specifically for 
mcciical education. Dr. Starkweather 



suggested four specifications for any 
such new language. 

1. The language should offer ease of 
entry to the computer novice. In connec- 
tion with this, he stressed the extremely 
poor quality and total unreadability of 
existing manufacturer produced pro- 
gramming manuals, 

2. The language should offer ease of 
editings that is, facilit}^ with which the 
author can make minor alterations in his 
text and instructions. 

3. The language should offer case of 
translation from similar or more primi- 
tive programming languages. This is ex- 
tremely important so that instructional 
material can be exchanged between 
medical users in different institutions. 

4. The new language should offer the 
ability to print for the user his source 
level instructions utilizing more than 
one level of readability. This would 
mean that the individual relatively unfa- 
miliar with the program or even other 
programming languages could request a 
‘'print out of the program"* in a relatively 
conversational, even verbose, form. On 
the other hand, the experienced individ- 
ual should be able to request a state- 
ment of the program in very much more 
terse, compact, and efficient style. 

A number of incmbcrs of the audience 
expressed their agreement with the re- 
quirements Dr. Starkweather laid out as 
well as the general Inadequacy of pres- 
ent programming languages. Ot\e mem- 
l>cr suggested that with resivocl to medi- 
cal education, vve were at a point analo- 
gous to the days of scientific program- 
ming before the existence of FORTRAN 
and other appropriate scientific high 
level languages. 

There was considerable discussion at 
differerrt |>ortiom in the afternoon meet- 
ing from the avidience concerning the 
recurrent theme: why arc there not 
more existing completed courses in mod- 
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(cinc available through computer-aided 
instruction. All partidpanU seemed in 
agreement: 

1. That there are no completed medical 

courses. 

2. That even the most promising provi- 

sional courses at the present will 
have to undergo extensive medical 
testing and evaluation. 

3. That an entire medical course, at 

least in the traditional medical 
school sense, is unquestionably far 
too large a unit with which to be- 
gin allempting computer-aided in- 
struction. Many participants fcU 
that a single one hour lecture 
might represent a realistic unit for 
the medical educator to approach 
with computer technics. Other par- 
ticipants, especially Dr. Greenes, 
preferred to approach a clinical 
problem rather than either a lec- 
ture or a course. The magnitude of 
the problem was hard to represent 
bvit nc agreed that it might well be 
of the same order of magnitude as 
a single Icotiuc. 

There was considerable discussion 
aimed at explaining the present cssci* 
lially primitive slate of computer-aided 
instruction In medicine. Many explana- 
tions were offered. A popular one 
seemed to be the belief that the prob- 
lems in medicine were essentially very 
much more complex than the problems 
presented in teaching undergraduate 
subjects of a seemingly more quantila- 
tivc and more rote nature. Examples 
were offered; statistics, languages, 
grammar school spelling and arithmetic, 
and college ph\Tilcs, The latter example 
was dialTengctl by one of the panel 
members who suggested that U had cost 
more than $ 200 ,(l 60 to program one col- 
lege ph)‘svcj course, and that the final 
iTSult had not been totally acceptable to 
the teaching department involved. Con- 
tcquenlly, one was left with the feeling 



that the participants generally agreed 
computer-aided instruction could be 
utilizied in areas where the teaching 
unit was sufficiently small, and the 
material to be covered was sufficiently 
well defined; but that in fact, neither of 
these conditions had yet been identified 
in a truly medical setting. 

As an extension of the above, there 
was considerable discussion from both 
the audience and the panel concerning 
the question of new roles for computers 
in medical education. Virtually cvcr)-onc 
was willing to concede that the use of 
computers to compile and give iiu^uiry 
capability to the patient record keeping 
system probably provided a suitable as- 
set in a medical cdvicalion setting. There 
w'as, howxvcr, very little enthusiasm 
from the audience to pursue this prob- 
lerii during the meeting. There seemed 
to he willingness to believe that this en- 
deavor was worthwhile, especially if the 
questions asked such a computer data 
base were aimed at an epidemiological 
or population level, and especially if 
epiacmiologisls were able to play a role 
in leaching students to ask suon ques- 
tions. 

In a similar w*ay, there w'as a great 
willingness on the part of the partici- 
pants to accept the contention that ov- 
ertly instructional, traditional 
computer-aided Instruction technics 
could lake their place in the long run as 
a valid portion of the medical education 
armamentorium. Even so, most |>coplo 
were willing to admit that accomplish- 
ments in this sphere were presently 
quite limited and the technics csscnlially 
still primitive. For this reason, there w as 
a verv geT.eral consensus that llio profes- 
sion should continue vigorously to seek 
new imaginative applications for com- 
pulers in teaching. This might take the 
ulCmatc form of diagnostic games, com- 
puters as an aid in logic and reasoning, 
computer simulation of mental and/or 
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physical processes or as yet totally un- 
discovered applications. In order to pur- 
sue this course of action, it was agreed 
that one should make every attempt to 
include objective evaluation criteria into 
such experiments. It was the belief of 
the section chairman, to which he gar- 
nered surprisingly little support, that the 
proper application of cost effectiveness 
criteria cannot be made to the experi- 
mental systems unless one carefully 
specifics the slates of development 
through which such S)*stems need to 
pass. In this sense, he was lu-ging the 
application of staged development tech- 
nics utilized in engineering; lor example, 
creation of spccmeallons and require- 
ments, pilot model, prolot)pc model, 
test model, production model, and final 
evaluation. In defense of this contention, 
it should be noted that there was only 
one engineer in this section meeting. He 
was an industrial engineer and con- 
curred with the dcsir ability of staging 
experimeuts. 

A menil>cr of the audience asked the 
group for general recommendations for 
a viniversity administrator in planning 
for computer Icrniina) capability in the 
design of a new campus strucltirc. llic 
consensus was that; 

]. The school should make provision for 

E ossiblc coaxial cable connections 
ctxvccn the new building and the 
campus computer site (probably 



this would mean a tunnel). 

2. With tbe new building, suitable tele- 

phone line conduit should be pro- 
vided to each office and laboratory. 

3. False or “deaf” ceilings should be 

used. 

4. Air conditioning should be provided. 

5. No cables or electrical connections 

should be buried in concrete, nor 
should “channels” in concrete be 
used. 

These were general recommendations. 
The group agreed that provision might 
be made for a small peripheral comput- 
er in the new building if a large number 
(more than eight or twelve) computer 
terminals were planned. A small com 
pulcr would also probably be needed 
within the building if complex physio- 
logical simulations were planned (such 
as the automated anesthesia dummy de- 
scribed by Dr. Abrahamson). 

Inquiry was made of the group for a 
published estimate of national, stale, or 
regional manpower needs with respect 
to medical computer personnel. Primari- 
ly the question here was the need for 
development of medically oriented com- 
puter systems and curators, computer 
programmers, and information scientists. 
All those asked felt most such persons 
were in critically short supply but could 
neither document this nc^ nor identify 
institutions at which such persons arc 
being trained. 



In summary, the |xirticipants were enthusiastic about the potential of contri- 
butions of computers to medical education, they scorned to feel generally that 
this field is presently in its infancy; that it should be explored in an openly exper- 
imental framcvvoik. A major limitation to progress In the field at present Is the 
cxborbitar\t amount of autnor/clinician lime rc<|uircd for the creation of even ex- 
perimental s)'stems. The participants seemed to feel that this problem was best 
attac4ced by the creation of new computer programming languages, pr(>lan- 
guages and/oT systems, of a sort to support conversatiorial mode interaction be- 
tween the relatively inexperienced user of computers and the necessarily large 
and complex computing systems which will bo required. 



COMPUTERS IN CONTINUING 
MEDICAL EDUCATION 

GEORGE E. MILLER, M.D.' 



I will begin with a disclaimer My 
knowledge of coin pv ter s is best de- 
scribed as rudimentary, and if I sneak 
willi any semblance of confidence abeut 
bits and core, of CRT and 360*50, it is 
because my associates have pro- 
grammed me with some care to respond 
appropriately to natural language input. 
Having made such a confession I must 
also acknowledge that my justification 
for accepting an invitation to appear on 
this program is based upon a conclusion 
that the key word in context of the 
conference title Is not computers but cd* 
ucalion, a field in which 1 feel some- 
what more comforlable. 

Kor in fact, the computer is neither 
more nor less than a tool, neither laud- 
able nor objectionable except in terms of 
the task it is to perform. And when that 
task is continuing education llicn the 
problem Is formidable, for debates 
nlx>ut the [)crplcxmg and frustrating N- 
siies in this field grow ever more shrill. 
Of only one thing do wo seem certain; 
the pressure is moiinling for the univer- 
sities and others to do something, piefer- 
alily soni ething new and hopefully wilh 
rusv tools as well. Among the new tools 
to u hich we arc turning, the computer Is 
nearing the crest of a wave of rnlhti- 
stasri! that was only recently vacated by 
programmed instruction. Oiscu^sion of 
ctunputer assisted instruction Is l>ccom- 
ing a babel In the land, and talk of net* 
work^ and compatibilities and softuarc 



can be heard almost everywhere. But in 
the language of communication thconsls 
the educator must ask whether all this 
sound represents a message ~ or merely 
noise. 

I am personally inclined toward the 
view that there is not only an important 
but a critical message to be heard, but 
one cannot deny that the noiscniakcrs 
arc also plentiful among us. These arc 
the ones who seem to be saying that this 
dazzling new medium is the message ~ 
and like the medicat educators before 
them who m.adc the same claims about 
films and television and programmed in- 
struction they divert us Irom the central 
c^ucslion of educational purpose to be 
served by such exciting tools. 

This cautionary comment must have a 
familiar ring, but it is now touched with 
a note of urgency. For the widespread 
restlessness about Improving means of 
bringing an ever growing body of medi- 
cal information more rapidly to praetj* 

I loners is accompanied by a mounting 
faith that the incredible inform alien 
storage and retrieval capacity of the 
computer can accomplish this allegedly 
critical task, ^\'hilc it would l>c foolh.ir- 
dy to deny the accelerating pace of 
knowledge growth, it Is at least dcfciwi- 
ble to raise the cpiestion of whether the 
problem of eontimiing cdueatioii is less- 
ened or magnified when a practitioner 
seeking information gels a StKnL.M\S 
printout of eighty citations on the topic 
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about which he has inquired. The tech- 
nical achievement of suA a production is 
staggering, but the educational impact 
may leave more than a little to be de- 
sired. 

The educational issue then is not com- 
puter capacity and versatility, but rather 
what kind of information shall be stored, 
in what manner it should be processed 
and retrieved, in order to serve what ed- 
ucational objectives. In my o\yn trou- 
bled search for an answer to this ques- 
tion I have conie, on April 4, 196S, to a 
point of identifying four functions to 
which the computer seems both ready 
and able to make a contribution to con- 
tinuing medical education. I share them 
with you not in any confidence that they 
will survive your scrutiny, but In the 
hope that they may serve as organizers 
for our discussions this afternoon. 

The first, to which I have already rc- 
fened in passing and which will be 
more fully developed by Dr. Cummings, 
is the library function. I find it difficult 
to believe, as some seem to Imply, that 
the library of the future will l>c a collec- 
tion of discs and tapes and punch cards 
from which words will flow’ as a steady 
electronic stream in response to the 
command of a discriminating client. But 
the evidence is persuasive that there are 
more efficient means of searching for an 
informational bit than poring through 
a card catalogue or the Index Medicos. 
Ilic computerized search function, with 
retrieval of selected abstracts or lapid 
reproduction of full texts will surely be 
an imjHutant conlribulion to the physi- 
cian who has a sjKcific question for 
which a specific and documented an- 
swer is available. Aside from the techni- 
cal problem this |>oscs, there is an edu- 
cational problem of retrieval, of match- 
ing level of inquiry with level of rc- 
5 {X>nsc, and not ovTrvvhelming the in- 
quirer with more information alKUit jKn- 
guins than he cares to have. I susjKCt 



that this library function will be of 
greater significance in the continuing 
education of academicians (who seem 
to have a high level of tolerance for ir- 
relevant information) than for practi- 
tioners — but that is a matter of specula- 
tion. 

The fact remains, however, that this 
function will be useful in fulfillment of 
only one objective of continuing educa- 
tion, and one which at least some of us 
believe to be of secondary rather than 
primary importance; that of prodding 
information. It docs not guide learning, 
but responds to praclilioncr-gcncratcd 
inquiry w’hich can as easily be a major 
effort to find small things as essential 
study of critical matters. Such random 
activity must sometimes be best de- 
scribed by Dorothy Parkers phrase, 
“Ihcfc is less hero laan meets the eye.” 

Secondly the computer can serve a 
corwu/fflriio function. It has been said 
by some, although without benefit of 
any discernible evidence, that the best 
continuing medical education occurs 
when a consultant and practitioner work 
together in solving a jicrplexing clinical 
problem. To the extent that this is true, 
the role of the consultanl-leaehcr would 
appear to be primarily that of bringing a 
special body of in formation, or a partic- 
ular skill in appl)ing both familiar and 
new knovvlcdec, to the problem at hand. 
It is such a function tnat seems to be 
served by some of the iniagmalive com- 
puter applications that have already 
been inlToducctl into clinical practice in 
some parts of the coimlry. For example, 
the computer analysis of electro eardio- 
grams which not only frees a physician- 
inteqirctcr from routine tasks but also 
foaises his attention u[x>n elements of 
the tracing that rcmiire sjx^ial consider- 
ation; the clinical diagnostic system 
which guides a practitioners inqirir)’ 
and from a va^l mernor)' core can sug- 
gest diagnostic probabilities, as well as 
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possibilities that have escaped his atten- 
tion; the rapid calculation of radiothera- 
peutic dosage which incorporates con- 
sideration of many variables that might 
olhcpivisc have been neglected or omit- 
ted; the continuous monitoring and syn- 
thesis of physiologic data from patients 
in coronary care units, patients whose 
course may be modified by early recog- 
nition of impending complications; the 
rapid analysis and weighting of histori- 
cal items in screening examinations, 
with probabi’.ily projections that allow a 
practitioner to focus his attention upon 
the most significant matters In his cn- 
counler witn nn individual patient. All 
of these may provide substantial and 
significant assistance to a physician in 
the fulfillment of his professional role, 
and thus contribute to solution of the 
critical manpower shortage which besets 
us. hut the (jt*eslion that must occupy 
our allenlion here is whether these 
things, or indeed whether more conven- 
tional consultative exchanges, arc educa- 
tional in the sense that they extend the 
learner’s ability to deal more efficiently 
or more effectively with comparable 
problems he cncovmtcrs in the future, or 
in another setting. It is surely possible 
that they may do so, and I am prepared 
to iaccepl this as a [>otcntial, if not a de- 
monstrably achieved outcome. But at 
the same time it seems Imjwrlanl to note 
that a cnitch may lead to dci^ndcncc as 
well as to independence. Nor docs the 
use of a more efficient bind of crutch 
necessarily influence the nature of the 
ovitcomc. Tlicre would probably l/c no 
dispute with the \*ic\v that electronic 
incasiirciiicnt of the hemoglobin lexxl Ss 
more precise than the Tahujuvsl method, 
but there might l>e considerable ques- 
tion wliether the In trod notion of accura- 
C)‘ has led practilioncTS to more rational 
pr.^cticcs In the treatment of anemia. 

If the comultalion function is to be 
useful as an educational instrument it 



must probably be accompanied by some 
systematic review of what has been 
learned, not merely what diagnostic or 
therapeutic goals have been achieved. 

Third is the iuiorial furxfion, perhaps 
the most familiar of the species included 
under the genus computer assisted in- 
struction. It is this form Lliat many u- 
sionary directors of continuing educa- 
tion programs have in mind as they pic- 
" ‘je a computer network for a city, a re- 
gion, a state, or in the ease of El^U- 
CO.\f, a nation. Such a mode lends itself 
to discrete course packaging, and to sys- 
tematic consideration ot highly inipor- 
fanl clinical content and concepts. \Vith 
the complex branching capabilities 
which even a simple computer offers, 
the instruction may proceed not only at 
an indMdual learner’s pace, but may 
b)^^ass informational bits with which he 
Is already familiar. Unlike the book, 
which also offers these advantages over 
classroom instruction, the tutorial fonn 
of computer instruction forces active 
participation in the learning process and 
some assurance that the ncocsiary con- 
tent has been mastered to n predeter- 
mined level before proceeding to the 
next stage. And the recitation, which 
most such programs require, has the ad- 
vantage of being personal and private so 
that the revelation of ignorance carries 
no social threat, unless the learner per- 
sonalircs the compulcrlutor and tries to 
beat the transistor! in much the same 
fashion that students have for so many 
)cars tried to beat the Ji vang teacher. 

liccogniring these several ad > ant ages, 
which reflect the (ncor^xiration into edu- 
cational Ic. lino logy of substantiated 
ibcorclical concepts of learning, there 
arc still man) v ho cry out in despair 
that sncIi mechanical and dispassionate 
inslnietion mig''* displace the warm and 
personal interaction between live teach- 
er and live student. I susixxt that most 
such complaints come from teachers pio- 
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turing what they think they do with stu- 
dents rather than from any accurate de- 
piction of what occurs in lectures, 
conferences, even seminars, in hundreds 
of continuing education programs all 
over the nation. Is listening with one 
hundred others to an expert talk (with 
the opportunity to ask questions at the 
end of the hour — if there is lime left) 
really personal instruction, while engag- 
ing in active indi\ndual analysis, in- 
terpretation and discussion of informa- 
tion with a carefully programmed com- 
puter really impersonal? And for those 
who insist that such an exchange is mere 
training, not real teaching, I would note 
the comment by B. F. Skinner in a re- 
cent Science paper that uhen we know 
what we arc doing it is called training, 
and when \vc don’t it is called leaching. 

Despite the advantages of planned 
coverage, carefully specified Icaniing 
objectives, active learner participation 
and immediate feedback, the tutorial 
mode of computer assisted instruction is 
still focused upon information which has 
been selected and processed by a teach- 
er, and whose acquisition is assumed to 
be accompanied by transfer into profes- 
sional practice. It is this assumption that 
must 1^ most seriously qviestioncd, for 
practitioners have long knoum, for ex- 
ample, ibat bacteriologic identification 
of an organism should precede institu- 
tion of antibiotic therapy or, at the very 
least, confirm the choice after the fact, 
but even casual iiisj>oction of the reality 
reveals the frequency with which this 
logical pattern is ignored. Such discrep- 
ancy between knowing and doing is the 
most disquieting problem of all for 
tboughtful leaders in continuing c<luca- 
tion. 

It is for ibis reason that I am particvi- 
larly attractcil to the fovirtl* educational 
function the computer may play - the 
functiou. In order to avoid 
any potential argument let it l>e oixnly 



acknowledged at once that simulation is 
not reality. But at the same time let the 
claim be underscored that for some in- 
struction it may be more useful than 
reality. For in simulation it is possibl3 to 
extract a critical and planned segment of 
an often r^omplox hiuI iinpredietable 
world; to control the evolution or se- 
quence of important variables; to pro- 
vide learners with opportunity for inde- 
pendent action without doing harm to 
either subject or object; and through 
this d)Tiamic mechanism to provide 
immediate feedback of consequena's of 
an intervention, not merely information 
about what might transpire. It is an in- 
structional form that focuses upon use 
rather than mere possession of knowl- 
edge. 

There arc already exciting develop- 
ments in computer simulation, ranging 
from relatively simple to dismayingly 
complex. SpiN'ack's arrh)lhmia trainer, 
one of the simpler applications, creates a 
situation in which sudden changes in 
electrocardiographic pattern displayed 
on an oscilloscope demand prompt and 
proper application of electrical stimuli 
to a s)Titnctic torso in order to reverse, 
for example, ventricular fibrillation or to 
initiate appropriate pacing of a jx>tcn- 
tinlly disastrous heart block. A more 
complex mode is Sim 1, Stephen Abra- 
hamsons* life-size dummy whose respira- 
tory activity, pulse rale, blood pressure, 
skin color and pupillary size, among oth* 
or things, arc under computer control 
and respond to drug administration as 
well as to other interventions used by 
the anesthesiologist in managing pa- 
tients at surgery. Using this model, a 
trainee can he faced with a variety of 
situations he will shortly encounter in 
the ojKrating room, and allowed to deal 
with them re j>ca led ly, without risk to 
real patients, and with immediate feed- 
back a1x>ut the effect of his judgments 
and his actions. 



Another program is John Starkweath- 
er's simulation of the clinical interview 
In which a student may play either the 
role of physician or of patient, while the 
computer carries out Ino other part of 
the dialogue. 

The method with which I am most fa- 
niliar, however, Is that under develop- 
ment in our own unit under the direc- 
tion of ^^itcheH Schorow. This has 
grown out of the printed patient man- 
agement problem, the erasure examina- 
tion whlcn has become one of the more 
familiar simulation methods in current 
medical education. Here the learner is 
provided with the bare bones of a prob- 
lem — usually little more than the pa- 
tient's presenting complaints. Selection 
of general problem-solving strategies 
and specific management tactics must 
then he independently generated using 
natural language Input, and with re- 
sponses provided through typed output, 
V'isual display (such as x-ray, electrocar- 
diogram, biopsy specimen), or sound 
(such as heart sounds, lung sounds, and 
segments of spoken historical Informa- 
tion that provide non-verbal as well as 
verbal cues). Management interventions 
are fol lowed by alteration in clitdeal 
coursCr sueli that a learner may deal suc- 
cessfully with a problem, inadvertently 
Induce complications with which he 
must then deal, or lose the patient to an- 
other physician — Or to the pathologist. 
But l^rarus-li^c, this patient can rise 
from the dead, and if necessary suffer 
the same fate again and again. 

While such an encounter Is demon- 
strably exciting, we must certainly avoid 
the comfortable trap of equating excite- 
ment with loamfng. They mav oe asso- 
ciated, but it Is not necessarily so. The 
carefully created simulation, however, 
docs provide an unusual opportunity to 
determine the means by which a learner 
attacks a problem, the t>pe$ of oidence 
he accumulates Wforc taking action, the 



relative weight he assigns to evidence of 
different kinds, the extent to which ef- 
fects of early interventions Influence his 
subsequent actions, and hIs manner of 
dealing with deviations from anticipated 
outcomes, if the sampling of this behav- 
ior is sufficiently large it may even allow 
some generalization about his chaiacter- 
istic pattern of performance which, 
when cast against the judgments of a 
criterion group, can focus attention 
sharply upon the strengths and weak- 
nesses of his know ledge and his methods 
of utilizing knowledge. In short, it pro- 
vides diagnostic data which mav then be 
used to prescribe educational therapy 
specifically selected to correct his per- 
formance deficits — therapy that may be 
In the form of books, journal articles, 
courses or tutoring. 

It is such a posture that has been as- 
sumed by the EDUCOM Task Force on 
Continuing Education — that education- 
al therapy abounds, but diagnosis of 
need is in very short supply. With the 
Educational Testing Service and re- 
search groups In medicine, criginccring 
and education a proposal has been de- 
veloped to study use of the computer, 
among other methods, for the purpose 
of developing such educational diagno- 
sis, and not merely the shotgun therapy 
which scorns to characterize so much of 
continuing education. It Is the hope of 
this study that a library of simulations 
designed to pioWde the basis for diag- 
nosis will not only direct attention to in- 
diWdual and group needs but will also 
heighten the awareness that learning bc- 
iris with recognizing a question, not 
aving an answer, with the process of 
seeVir.g and not alone ssilh some prod- 
uct that others have found. 

Such ft shift In educational strategy 
will not be easy to achieve, for teachers 
have long managed to divert a variety of 
instructional methods to their own 
needs. There Is no reason to bv'licve that 
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a compute! will accomplish a magical 
traaasformalion of teacher-centered in- 
struction to student-centered learning. 
But it does create an opportunity for In- 
dividuali^aUo^ of learning that one can 
hope will be grasped and exploited to 
the full. 

The developmental stage of any new 
method is certainly not the time to raise 
questions about cost, yet it Is an issue 
that must ultimately be faced both real- 
istically and quantitatively. The cunent 
enchantment with compuleis not 
last forever, and when the enormous 
bills for programmers and analysts, long 
lines and equipment begin to seem bur- 
densome, we had belter be prepared to 
defend with something more solid than 
enthusiasm our view that computer-aided 
learning for practicing professionals is 
not Only effective but competitive. As 



the Investment in hardware and soft- 
ware goes steadily upward, I would 
plead for the diversion of some signifi- 
cant segment of that investment. In ev- 
ery developmental program, to system- 
atic study of use (once the bloom of 
novelty has faded), of cost (once the 
developmental price has been paid), but 
most importantly of effectiveness. And it 
i:> here that we will require a measure of 
humility that has not always character- 
ized the physicians who conceive pro- 
grams of continuing education, or the 
media masters who have nourished and 
supported them, Our greatest need is a 
spirit of Inquiry — not about computers 
(in continuing education), but about 
continuing educatton (witlj computers). 

And on that shrill note I shall retire to 
the wings. 
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COMPUTERS IN CONTINUING MEDICAL EDUCATION* 

Panel; George E. Miller, M.D., Chairman, William T, Blcssum, M,D., Edward N, Brandt, Jr., 
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The panel and audience were charged 
by its chairman with development of 
discussion and rccommondalions which 
could guide those interested ;ti working 
with computers in Continuing Medical 
Education and those who may have re- 
sponsibility for funding such work. 
Members of the audience introduced 
themselves and their major interests and 
rjiicslions. It was found that although 
the audience consisted of a diversity of 
backgrounds and interests, their ques- 
tions were very similar, The audience 
was actively involved fn the discussion 
and this report includes their contribu- 
tions as well as those of the panelists. 

The first major consideration was the 
Tclationsliip of computer based instruc- 
tional systems to olWr computer based 
health care systems. Dr. William Blcs- 
sum of the California College of Medi- 
cine jx)intcd out that he had l>ccn active 
In altcmpling to stimulate interest in 
computer based systems by pilot appli- 
cations for medical students. The com- 
puter group has been active for about 
nine months and there is inlcTCst in de- 
velopment of a system which utilizes 
computer-assisted instruction to the 
maximum c.xtcnt ]X)ssjblo, At the present 
time it is di^^lcult to decide on which In- 
structional s)^lem to Use since a number 
of significant ones arc available, Fur- 
thermore, there are a wide variety of 
terminals and philosophies of design for 
the operating s)*slem in which comput- 



er- assisted instruction takes place. U is a 
continuing problem to determine how 
best to take advantage of the various 
programs available because of the w^de 
variety of hardware configurations. Dr. 
Blcssum challenged those engaged in 
computer-assisted instruction to realize 
that it is in an cmbr)’onic slate of devel- 
opment and must be approached in an 
investigative atmosphere. He empha- 
sized Ine need (o be as compatible as 
possible and yet to realize that there is a 
danger In being compatible too early 
since this may restrict innovative ap- 
proaches. Therefore, he emphasized the 
need for generation of a master plan for 
all applications of computers in medi- 
cine so that library systems, medical in- 
formation systems, computer-aided in- 
slrueticn, etc. will all be components of 
the same general system, Tne advan- 
tages and disadvantages of such an ap- 
proach were discussed. Specifically, it 
was pointed out that building a comput- 
er based instructional sptem as a part 
of a total health information systerTi 
could provide a means for stimiwaJ^'ig 
physicians to acce[>t the computer as an 
instructional tool and to use it. The prin- 
cipal problem to l>c faced by devclojx'rs 
oi computer-aided instniction is how to 
get this form of instruction to physi- 
cians. Should terminals be pul in aoc- 
tors offices or can they take advantage 
of this through other means? It was felt 
that if computer-aided instmclion were 
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pari of a total health information sys- 
tem, one could more readily justify pro- 
viding terminals for physician's use since 
it would be possible to have at his dis- 
posal many functions, such as summa- 
ries of clinical data, diagnostic assistance, 
information retrieval, etc. that would be 
of value to him. Furthermore, this would 
provide a means for obtaining feedback 
from the physician so that he couM help 
define and contribute to the objectives 
of his own continuing education for 
translation by educators and computer 
seienlisls. The question as to whether 
physicians would use terminals for any 
purpose was discussed. It was alleged 
that medical students do accept them 
and if a computer based health informa- 
tion system were able to play a consulta- 
tive role and lead to improved patient 
care, physicians would make use of it. 
However, much discussion centered 
around the physicians acceptance of the 
computer esjx'cially as a part of his con- 
limiing education. In summary, there is 
virtu.illy no Information available as to 
physician acceptance and careful study 
is needed. A computer based health care 
system could provide high quality care 
of patients since it would have the ad- 
vantage of having necessary’ data avail- 
able and the results of group experi- 
ences to assist in judgments. However, 
in the development of such a system 
there is the need to separate judgments 
from facts and to provide for group 
rather than indmdviaf judgments. 

Another advantage to Introducing 
C.M by this means is that it would have 
the virtue of |>oer group pressure on the 
ph)’sieian to take advantage of a very 
potent tool. However, pliv-sieians v\i(l 
make use of it only if tnc computer 
based system improvrs the quality’ of 
their activities and if the system is well 
designed and meets their needs. All of 
this discussion was based upc>n accept- 
ance of the concept that the purpose of 



continuing medical education is to im- 
prove li;e quality of patient care. The 
group was also reminded that knowl- 
edge of what constitutes good care and 
use of such knowledge are not necessari- 
ly the same. 

The disadvantages of building the 
CAI sy’stem as a part of a health infor- 
mation system were also discussed. In 
the first place, such a system may be- 
come a "crutch'" for the physician and 
thereby defeat thinking and learning. 
Furthermore, if the system provides for 
punitive action or enforcement of partic- 
ular ideas, this may in itself defeat its 
acceptance and lead to the development 
of poor altitudes towards its use. How- 
ever, such a system permits continuing 
education to be indirect and thereby not 
require the physician to spend blocks of 
time in his education and this is desira- 
ble. Furthermore, the material presented 
will be relevant to the physician's aelivi- 
tics and this has the greatest chance of 
inducing bcha\*iora1 changes. 

The discussion then iarned to consid- 
eration of the objectives of a computer 
based continuing education program. 
Quality patient care was considered to 
l>c the most imivirtant objective of con- 
tinuing medical celucalion, but relatively 
little is knowTi alwut what motivates 
phy’sicians to obtain further eelucnlion. 
Studies should \yc carefully and system- 
atically designed to determine what the 
objectives of continuing education 
should l)c. A very important y>o\ni was 
made that getting information to physi- 
cians was not as imjK>rlant as inducing 
them to make use of this information. 
The material presented should bo rele- 
vant to his Own goals and therefore it Is 
necessary’ to determine vs hat these goals 
are and to translate them into meaningful 
educational strategies. The evaluation of 
any instructional system must depend 
upon the realization of goals and there- 
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fore is necessary to set objectives in 
advance. 

Otb^r questions that need careful and 
systematic study were also discussed. 
For example, doer a computer based in- 
structional system actually inhibit learn- 
ing or perhaps belter staled, under what 
conditions ciocs such a system inhibit 
learning? If, for example, it were used 
by the physician as a decision maldng 
tool, it might influence the physician to 
not continue his cducatio’a. Further- 
more, its use as a policing device would 
lead to rejection raihcr than stimulation. 

Is it belter to introduce computer 
based instruction into medical scnools 
and then continue with these students 
than to try to expose physicians who 
have developed study habits over years 
of training? The general problem of In- 
fluencing medical students and ph)'si- 
eians to change their habits of study Is 
faced by educators Inloreslod In CAl 
since it will not be used if it does not 
prove lo l>o a more efficient and success- 
ful method. CAI docs have the Wrtuc 
— lially of providing individually 
Instruction and thereby of being 
an effective self study tool. However, 
the realiration of Ibis depends upon ac- 
cess to medical Infornriation on a very 
side scale, hence, the apparent desirabil- 
ity of mating it a part of a larger infor- 
mation system, 

Disctission then tumcil lo the role of 
computers in continuing medical educa- 
tion. The tutorial approach may not be 
indicated in conllnuing medical educa- 
tion due to the potential lack of rele- 
vance to the phy$ician*s concern. Again, 
it was emphasired that the Immediate 
problem is not one of getting Informa- 
tion ti) ibe physicians, but of stimulating 
Whav-Ioral changes based upon that In- 
fojmalion. The tutorial approach is most 
effective In the presentation of factual 
information lo b? learned and hence by 



its very nature is fairly rigid in design. 
Flexibility to permit the student to se- 
lect the material or portion thereof that 
he is lo learn is very desirable. More so- 
hislicated diagnostic methods to assist 
im in this selection would be of benefit 
but at this time such methods have not 
been developed. Computer simulation of 
clinical situations that are rcvelanl is one 
approach lo gel the physician inv^olved 
and to make it important to his own 
practice. Such situations permit the phy- 
sician lo develop a solution with the as- 
sistance of the computer as a tutor. Ex- 
periments involving simulation of clini- 
cal situations arc underway, hence, more 
information about this aspect will be 
forthcoming. Again, the consultative 
role of provi’ding information and assist- 
ance in decision making by means of 
tie-up with a health information system 
was also pointed otit. This provides the 
ph)*sioian with access to empirical as 
well as theoretical information and ex- 
periences, 

TIjc consensus of the panel was that 
the goals of continuing medical educa- 
tion are lo develop a set of attitudes to- 
wards: (a) use of records in the broad 
sense as a basis for judgments; and (b) 
appropriate medical care. 

Therefore, it is recommended that 
there bo clevelo|X‘d carefully and sys- 
tematically planned investigations into: 

(a) what should l>e the specific objec- 
tives of continuing medical education; 

(b) will Instruction developed from a 
computer based health information sys- 
tem enhance or Inhibit learning; (c) 
how will physicians use such s)^s terns; 
(d) will such s)’stems lead to increased 
motivation for further education; (e) 
how docs one dcvrlop compatibility of 
existing systems; and (f) wbat arc the 
jxjlentia! uses of compvilef-aldcd in- 
slniclioii in continuing medical educa- 
tion? It was stressed that work lo dcvel- 
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op a greater diversity of computer based 
instruction must continue and be evalu- 
ated before the role of the computer in 
continuing medical education can be de- 



fined. However, such development 
should not ignore, but should attempt to 
gather information to answer the above 
questions. 
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THE CHALLENGE 

An important purpose of medical edu- 
cation is the transfer of sldlls, knowl- 
edge, and information from a variety of 
sources through a variety of media to 
the student and practitioner. A most Im- 
portant resource in this process is the 
medical library. \Vc have become In- 
creasingly conscious of the relative roles 
of other media so important in the com- 
munication processes, including the in- 
formation transfer that lakes place 
through personal discussions \«lh col- 
leagues, allcndancc at professional lec- 
tures, and the use of Ine newer audio 
and audio-\isual materials including 
teleusion. But the literature — the wtU- 
Icn word — remains as a principal vehi- 
cle through which scientific concepts, 
facts, ideas and observations are trans- 
ferred. 

The growth of the medical literature 
has almost doubled over the past dec- 
ade, as has been true in practically all 
scientific fields. The literature itself 
abounds with articles discussing the ex- 
ponential rate of growth of scientific 
periodicals and \\ti lings. The order of 
magnitude of this “explosion" has been 
fre<)uenlly debated with estimates of 
scientific periodicals being published to- 
day ranging from ii5,000 to over 100,000. 



It bris been estimated that there are 
more than 18,000 serial publications in 
the biomedical field alone, with an 
equal number of medical books and 
monographs generated throughout the 
world annually. The growth rate has 
been Jcscribca by Derek de Sol I a Price 
of Yale University. *'Thc most remark- 
able conclusion outained from the data 
. . . Is that the number of journals has 
grown exponentially rather than linear- 
ly. Instead of there being just so many 
new periodicals per year, the number 
has dovibled every so many years. The 
conslant involved is actually about fif- 
teen years for a doubling ..." (1) 
Vannevar Bush wtoIc as early as July 
1915 . . , “There Is a growing mountain 
of re search. But there Is Increased c\t- 
dence that wc arc being bowed down 
loda^ as specialization extends. The in- 
vestigator Is staggered by the findings 
and conclusions of thousands of other 
workers — conclusions which he cannot 
find lime to geasn, much less remember, 
as they appear." (2) 

Our society looks to the university as 
the instrument of primary responsibility 
for ihc creation of knowledge through 
research, its dissemination through 
teaching and Its application through 
service, ^fhe centers for education in the 
health professions have been developed 
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within this university framework to ac- 
complish this task with respect to 
knowledge in health-related matters. So- 
ciety originally looked to the libraries as 
instruments to preserve recorded knowl- 
edge at a time when the corpus was 
small and manageable. Today they have 
taken as an equally important function 
the organization and retrieval of needed 
information from the present enormous 
body of knowledge. Thus the library 
represents a central learning resource 
serving our universities, 

Health science libraries are faced with 
an ever increasing challenge. This is not 
due solely to the rapid growth in the 
medical and related literature; the wid- 
ening range of users and their accompa- 
nying diversity of renuirements com- 
pound the problem, The list of users 
may include undergraduate students, 
graduate students, |X)sldoctoral fcllcws, 
interns and residents, members of the 
te.aching faculty in the basic sciences, 
clinical faculty members, practitioners 
in the health center or its environs, med- 
ical researchers, dentists, vctcriniarians. 
harmacists, nurses, and other allied 
ealth ^Krsonnel. It may also include 
other regional and hospital clients, 
health departments, welfare agencies, 
local industries, the county medical so- 
ciety, miscellaneous lay jKrsons, nttor- 
nc)-s and others. The basic homogeneity 
of the principal users of a medical li- 
brary remains as an asset to the meelical 
librarian in providing required serWees 
but as this list suggests, that advantage 
is no longet so pronounced. The impor- 
tance of In ‘crdisci pi inary application of 
knowledge and research will continue to 
require expanding services and capabili- 
ties of the medfeal library. 

In addition to the quantitative 
changes of information arrd users facing 
the medical libraries, there have l)cen 
lmix>iiant qualitative? changes in n>cdi- 
cal education. Curricula have broadened 



in scope and depth. New subject areas 
have been incorporated as knowledge 
increases. Under^aduates are more in- 
volved in research. The student must 
absorb more of the total body of science 
and then be rcsj>onsible for keeping up 
with new Imowledge. It is no longer the 
medical undergraduate alone who must 
be educated, but the practicing physi- 
cian. B, V. Dryer in the article ‘‘Lifetime 
Learning for Physicians** has slated that 
. . . “The gap between scientific knowl- 
edge and application grows wider each 
year because of at least four reasons: 
the rapid advances of research; the mal- 
distribution of opportunities for contin- 
uing medical education; educational in- 
adequacies even in those places where 
opjx)rlunitics do exist; and patterns of 
educational organization and dissemina- 
tion of knowledge which are not effi- 
cient in terms of the physicians students* 
needs.** (3) 

The opportunity to prepare students 
with the needed attitudes to meet the 
later demands of continuing education 
must not lx? lost. The things which 
young medical students learn in their 
formal educational courses set the direc- 
tion of their future lives as physicians 
and scientists, Harvey Cushing, who on 
many occasions jx>inted out the linpor- 
tanec of the meaical library for sup[X>rl 
of mc-dical eilucation, research and prac- 
tice, commented on the importance of 
teaching students the value of their li- 
brary as follows; *"To inoculate a doctor 
with the lil>rary*habit he must bo caught 
young and here, as 1 sec it, you have an 
excellent opportunity in l)cing nearer to 
a growing antre of medical education. 

‘If )^u are to infect the young with 
the reading habit, you must set the trap 
for them, so bailed that they will walic 
into it unawares. Books must be made 
ftccrssiblo. It is somconc*s business in 
ever)* medical school to leach laboratory 
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methods to the students but it is no 
one’s particular business to teach them 
how to use medical literattire. . , (4) 

The subject of the need for continuing 
education should not be left without 
some reference to the place the hospital 
is assuming and the role of its library in 
this process. The hospital represents the 
location where ll*e professional health 
personnel c^n bring scientific knowh 
edge to bear on a problem. Within the 
last four decades, additional educational 
rograms have been started within the 
ospital as it represents an unique envi- 
ronment where certain Icinds of post- 
graduate education can take place. It is 
clear that the hospital must take increas- 
ing responsibility for education at all 
levels. ^Vhen the hospital is thus %iewcd 
as a scholarly insKlulion involved in re- 
search and education as well as patient 
care, the need for effective access to 
medical and related sdcnlific knowl- 
edge Is sclf-CN’idont. The dilemma Is that 
the requirement for iSe availability of 
the scholarly rec'ord Is so broad that no 
single hospital library has the resources 
to proi>crly respond. 

The problems of scientific communi- 
cation confrontinc; medical librarians ap- 
pear to approacn the Insunnounlabfe; 
quantity ot information rapidly growing, 
quality of education and research im- 
prosing, information user groups ex- 
panding, interdisciplinary requirements 
enlarging, and education throughout a 
professionaVs career demanding support. 
It seems likely that the health sciences 
libraries will have to take on a nesv iden- 
tity and function to meet this challenge. 

Ubrarics arc esen now rapidly turn- 
ing to the electronic tcchnoWy to re- 
spond to society's demands. ’Tre use of 
advanced data processing equipment, 
Including computers, coupled with the 
research a d techniques aevelopcd and 
being extended In the emerging field of 
Information sciences holds great promise 



for the solution of the biomedical com- 
munications dilemma. There is a warn- 
ing, however, that must be sounded be- 
fore exploring the applications and po- 
tentials of these new technologies. Wil- 
liam Osier, on stressing the importance 
of organizing and extending smaller li- 
braries in medical societies or hospitals 
for continuing education of physicians, 
wisely pointed out, "It is much simpler 
to buy books than to read them, and 
easier to read them than to absorb their 
contents.*' (5) In a similar v^c^v, it is 
much easier to introdijcc computers into 
libraries than it is to design systems for 
their productive and efficient use. 

//Is for teal Development of IJhrary 
Aufomoiion 

The mechanization of data processing, 
including both what is knowT) as EAM 
(Electronic Accounting Machinery) and 
the newer computer equipment, has an 
intricate and somewhat eUisfve history. 
As early as 17S0 Joseph Marie Jacquard 
invcnlctl a machine in which perforated 
cards controlled the pattern of hooks 
and needles for use In wearing; the 
English mathematician, Charles Bab- 
bage, conceived the idea of an automat- 
ic digital computer al>oi)t JS35. He 
called his proposed machine an *^anaMi- 
cal engine", but was unable to complete 
a world ng model before he died In 1871. 
(6) It is generally acknowledged that 
the first operational dcrico using perfo- 
rated cards for tabulation and statistical 
purposes was developed In the 187C’s, 
and represents the real genesis of the 
present multi-million-donar data pro- 
cessing industry. 

The insTnJon of the electrical tabu- 
lating machine using punched cards is 
generally altributcd to Herman Holler- 
ith, a statistician, who was ui>r)nng at 
that time for the United States Census 
Office. But from his own records, and 
those of others, it is clear that it wa.s a 



physician who first suggested to Holler- 
ith that an electrical machine utilizing 
punched cards should be developed for 
use In the 1880 census. This physician 
was Dr. John Shaw Billings, the princi- 
pal figure in the history of the present- 
day National Library of Nfedicine, serv- 
ing as its first director from 1865 to 
1895, and unquestionably the pre- 
eminent iigure in the entire field of 
medical bbrarianship. 

Serving as a consultant to the Census 
Office, Billings suggested in 1880 "that 
the various statistieal data of the living 
and the decedent might be recorded on 
a sinpic card by punning small holes in 
it and that these cards might then be as- 
sorted and counted by mechanical 
means according to any selected groups 
of these perforations.*" (7) I1ic electrical 
tabulating machine developed as a re- 
sult of these suggestions was actually 
used In connection with the 1886 mortaf- 
lly records of the city of Baltimore, in 
the Surgeon Generals Office for compil- 
ing and tabulating Army health statistics 
in 1889, and in the handling of data col- 
lected In the Uiailcd Slates Census of 
1890. 

The extension of these earliest devel- 
opments to library applications was 
non-existent until the middle of the 20lh 
Century. It is true, however, that me- 
chanical means for expediting library 
work have been consiaered for many 
years, although not in what would gen- 
erally be considered the data processing 
area. The very first volume of the fore- 
runner of the BuUetin of the Mc<l{col Li- 
brary Associaffon, the Medical Library 
and Hisiorkal Journal, dated 1903, con- 
tained an article on the use of the "t)pc- 
wxiting machine* In cataloging. In addi- 
tion, telephones have long been accept* 
cd as library machines, and, except for 
unusual items, hand bookbinding has 
given way to machine- bound books. It 
was not until the late 19J0*s, howevTr, 



that a few libraries began using unit rec- 
ord equipment for their various business 
operations such as purchasing and pay- 
roll. 

As computers became available, a few 
industrial and research libraries began 
limited information storage and retrieval 
work using these dcNices. Basically, 
these early systems closely resemble 
the processing done with the usual pay- 
roll and personnel records — formatted 
files, rcaa serially* The development of 
iho follov^ng generations of computer 
hardware and Improved software tech- 
niques have led to cxtcrvsi\o experimen- 
tal work searching for effective applica- 
tions to library operations. Probably the 
most significant advances in terms of the 
computer’s application to information 
storage and retrieval have been the tre- 
mendous increase in computer storage 
and the capability for a scries of users to 
query the central computer file simulta- 
neously and from remote locations. 
Tliese developments have opened a new 
era for the advancement of library ser\’- 
ices. 

Despite these applications and signifi- 
cant developmental aclmlies In the 
19^*s, library automation Is still In its 
Infancy. An exhaustive literature survey 
conducted bv Donald Black and Earl 
Farley for Inc publication of the first 
volume of the AnntijI Rctfno of Infer- 
malion Science and Technology (8) in- 
cluded very few articles covering as- 
pects of library automation systems in 
operation or planned In any single li- 
brary. The ON’crvvhclmIng preponder- 
ance of the literature consisted of why 
automation is the coming thing, why 
one should become familiar vsiln what 
machines can do, and discussed various 
applications to one library function or 
another. 

Aufomadon of Library Acikitki 

The word *s)*$lcm* as used In scientif- 
ic management or related to the scienlif- 



ic melliod of problem solution desig- 
nates an organization of interacting 
parts intended to accomplish a particu- 
lar purpose. This rather all-encompass- 
I'ng definition can include a wide variety’ 
of activities not the least among which is 
the library. The library is certainly in- 
tended to accomplish a particular pur- 
pose, or more accurately a scries of pur- 
poses, although these arc all loo often 
forgotten in the busy days of "doing 
things”. Certainly the library can also be 
properly considered an organization of 
interacting parts. 

System analysis is a method of han- 
dling complex problems — a method so 
simple and vet so powerful that it has 
been applied successfully to problems in 
engineering, business, and government. 
If a library is truly a system and system 
analysis has proved so valuable to other 
entcrpri'^cs, this method of problem so- 
lution should l>c equally successful for 
library operations. An examination of a 
library from the systems analysis Wevv- 
poinl is desirable before considering the 
mechanization of its acliWtics. 

Systems analysis demands the state- 
ment of both final obfectives and of in- 
termediate objectives. Othetwise there Is 
no way in wnich design and |>crform- 
ance can be measured and evaluated. 
Libraries may have many objectives or 
purposes. Thnc base been a numl>cr of 
efforts to stale these purposes in systems 
terms. It can be said, as V, W. Clapp 
has done, ... "to provide access in bilv 
liographic and physical terms to the rec- 
ords of human communication"; (9) or 
as staled by Becker and Hayes , . . "to 
obtain, preserve and make available the 
printed malcnals required for use by Sts 
patrons and In so doing to store record- 
ed knowledge for future generations.” 
(10) These expressions provide a back- 
ground against which each clement or 
part of the library system can be tested. 

The purposes and objectives of modi- 
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cal libraries have been outlined in the 
opeTiing paragraphs of this paper, par- 
ticularly the role of the medical library 
in medical education. Heal ih-reta ted ori- 
entation makes medical libraries dis- 
tinctive in that the user population Is 
considerably more homogeneous than in 
a general library, and the emphasis that 
has been placed on the library’s role in 
medical education rcmiires an active 
rather than passive role so typical of 
most libraries. These distinctions require 
careful consideration by those engaged 
in analysis, design and management of 
medical libraries. In spite of these 
differences, it is equally true that many 
of the detailed functions required within 
any library are basically similar. 

Oj^crationally, all library systems em- 
ploy four basic processes: Inc acquisi- 
tion of publications, the creation ot sur- 
rogates for ibc items, the storage of 
both, and the access to one or both upon 
request. In accomplishing this work, 
most library practice revolves around 
the records that are kept of the books, 
documents and periodicals that make up 
the main store of the library. The li- 
brary, thus, Is essentially a set of files. 
The basic set arc the items themselves; a 
second set arc those providing access to 
these items; and a third set arc the nec- 
essary administrative t)’ix? files. The 
creation and processing of these files 
and records presents an enormous data 
handling problem for the librarian. It is 
readily understandable that Dr. Billings, 
as a librarian, would be a leader in the 
development of mechanized data pro- 
cessing equipment, but equally puzzling 
that so little use was made of it throiicn 
the years by the libraries of the world. 

The first inventory of library automa- 
tion activilics was carried out in mid- 
19G0 by Creative Research Services, 
Inc., for the Document Division of the 
Special Library Association and the li- 
brary Tcchnolog)* Program of the Amcr- 
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ican Library Association. The report 
(11) tabulates the extent of mechaniza- 
tion and automation plans of 1130 insti- 
tutions of 6150 wUch replied to the mail 
questionnaire. 

The statistics resulting from this sur- 
vey indicate that the typical mechanized 
library is a university or special library 
with a collection of over 50,000 boob 
and more than 1,000 periodical titles. It 
has a staff of at least ten but no more 
than twenty members, evenly divided 
between professional and non-profes- 
sional. It nas its serial control function 
running on electrical accounting ma- 
cbinery ( E AM ) and Us accounting 
function running on computers. The fa- 
cilities of the library's host organization 
arc used ralbcr than the machine equip- 
ment owned or leased by the library it- 
self. This typical library has plans to 
mechanize its circulation control and 
accession list functions in the next one 
to two years. (This description of the 
Uplcal library was derived by Eugene 
D. Jackson from the statistics taken from 
the su^cy.) (12) 

Although such a {)^ica\ library Is only 
a creation from statistics compiled 
through a surscy and the data arc now 
almost two years old, it does give some 
indication of present conditions. Perhaps 
more meaningful arc the statistics showm 
in Figure 1 for the various library func- 
tions. The total 633 represents the num- 
ber of libraries reporting some use being 
made of titber electrical accounting ma- 
chinery (EAM) or automatic data pro- 
cessing (ADP) equipment. As shown by 
the percentage figures, progress has 
been made toward automating conven- 
tional library operations. 



First EAM and now a growing num- 
ber of ADP systems h^ve been adopted 
for the control of the library's manage- 
ment data in functions such as account- 
ing and in the processing activities rep- 
resented by acquisition, serials control, 
announcement fists and book eatalog 
and catalog card production. In the 
areas more dircetly related to service to 
the users, the progress toward automa- 
tion has been generally slower. This is 
the phase of operations involving the 
question of which documents in the li- 
brary contain information on a .specific 
subject. As Adkinson and Stearns have 
staled, (13) in the first phase (manage- 
ment data and processing activities) w'c 
have automated our inventory control 
over packages of information; in the sec- 
ond phase (the serWee to users), wc arc 
extending our efforts to automate Into 
the process of determining which pack- 
ages In the inventory contain answers to 
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particular questions. The function listed 
in Figure 1 as ^^retrospective searches* 
covers this latter activity and the per- 
centages of 12 and 24 shown in the table 
are higher than one might CKpcct, This is 
somewhat misleading when placed £n 
the restricted context of libraries; but is 
explained by the reminder that the sur- 
vey included the information sj^tems of 
industry where some of the more signifi- 
cant progress has been made In the au- 
tomation of retrieval operations. There 
are, of course, notable exceptions to this 
generalization, and in the medical field 
these exceptions exist largely because of 
the availability of the automated biblio- 
graphic citation file known as MED- 
LaKS (Kfcdical Literature Analysis and 
Retrieval System), eslabltshcr and 
maintained by the National Library of 
Medicine. 

Mcdic/il Library Applications of Auio- 
tnation 

The automation efforts at NLM were 
first directed toward support to the pub- 
lication of /ndex Affdicus. Beginning in 
1961, the Libra^ placed the biblio- 
graphic citations for the articles selected 
for entry Into Index Afcdioui on a mag- 
netic tape and used a computer to 
generate a printed oulput. Since August 
of that year, the printing has been ac- 
complished with the use of the comput- 
er file and NLM’s GRACE (Graphic 
Arts Composing Equipment), This same 
machine-readable file of citations now 
supjmrts the most significant and far- 
reaching example of computer applica- 
tion to rctro$pc<^vc searching of medi- 
cal literature. 

The MEDLARS file now contains ap- 
proximately 750,000 citations to articles 
from about 2400 selected medical Jour- 
nals. It can be entered through the te- 
Icctive use of the terms in a controlled 
s-ocabulary for the retrieval of the quali- 
f)ing citations. It requires trained 



searchers to transpose a user's question 
into the proper search formulation, the 
punching of a deck of cards for entry 
into the computer, and then the serial 
searching by the computer of the file for 
the citations. This is a batch computer 
operation, l.e., searches are batch^ In 
groups of 30 to 40 and are entered to- 
gether for a single pass against the total 
file, prnt-outs are made of the retrievals 
and the results forwarded to the reques- 
tor. 

The principal operating techniques of 
MEDLARS tor constructing and organ- 
Llng machine-readable records, and for 
the searching and retrieving from these 
records, still icquire the human to assign 
words as labels to represent the contents 
of the docun^enis. Tlico, upon searching, 
he must transpose a question into the 
same labels and express them in logical 
relationships in order to retrieve the 
item desired. Tlie human Intellect and 
ingenuity have been pul to the test to 
provide effective schemes by which this 
matching of terms describing llie query 
\nlh those iiscd as input descriptors can 
be accom plished. The controlled vocab- 
ulary uHd in iJic MEDMRS system Is 
an example of one of the more success- 
ful efforts (0 reach this goal This vocab- 
ulary consists of approximately 7000 
medical subject headings ( hfedioal Sub- 
[ccl Headings - hfeSH) and Includes a 
significant blorarclucal structure to as- 
sist in both indexing for input and as- 
Signlng tcims for rclrlcval. 

The NLM computer capability also 
support! the publication of a scries of 
products in addition to Index Medievs, 
The MEDLARS file is used to produce 
a monthly Bibliograi)hy of Medical Rr- 
lifirs, including articles from publica- 
tions such as the Anntwl Reifnc scries 
ard N’ufririon Rctfetr# and articles 
termed ‘'review" In such journals as the 
Sew England loutnal of Medicine. 
These arc of great value to students and 
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fo Individuals entering a new field of 
jnedical endeavor 

Retrospective literature searches that 
are accomplished in NfEDLABS are also 
published for general distribution when 
their subject matter is judged to be of 
relatively wide interest For example, 
NLM has published a literature search 
titled Computers in Medicine and Bfofo* 
gy that contains 327 citations to articles 
indexed for MEDLARS from mid-19d4 
through December 1967. A sampling of 
titles of other published searches in- 
cludes Tumors cf Cold-Blooded Verte- 
brates, Neoplasm Models, Clinical Stud- 
ies Invohing Estrogens, Epidemiology 
of Suicide, and Exercise arid the Elec- 
trocardiogram. 

The NLM Current Catalog Is a prod- 
uct of particular importance to the 
country's medical libraries. It Is issued 
on a biweekly basis and citations arc cu- 
mulated quarterly. The biweekly issues 
contain citations to publications cata- 
loged by NLM which have an imprint 
date of the current or two preceding 
years. Cumulations include citations 
to all publications cataloged by the li- 
brary, regardless of the dale of imprint, 
except for titles published before ISOl 
and certain other special publications. 
In accordance with the Library’s goal of 
improving and enhancing access to sig- 
nificant momcdical literature, the 19^ 
Current Catalog contains, for the first 
time, separately listed citations to tech, 
nical reports which have been acquired 
for the NLM collection. 

The National Library of Medicirc Is 
cuncnlly developing a complete auto- 
mated system rs a second generation to 
ibe present ^tEDLARS. The contractor 
is being selected to perform the s>*stem$ 
design, hardware Installation, and pro- 
gram Implementation - a project that is 
scheduled over a five-year period. The 
new $)* 5 tcm will meet future data pro- 
cessing and Information system require- 



ments for: 1) an increase in the level 
and speed of MEDLARS bibliographic 
services provided, including demand 
."{.marches and recurring bibliographies; 
2) an automated acquisitions and cata- 
loging system; 3) an on-line augmented 
Medical Subject Headings vocabulary to 
aid those responsible for indexing and 
searching; and 4) a drug literature pro- 
gram with search capabilities on chemi- 
cal compounds. In addition, the new 
equipment will be capable of expansion 
to accommodate computer processing 
required by a graphic image storage and 
retrieval system closely linked to the 
^iEDLARS search capability. 

Another development recently begun 
In computer application at NLM is the 
automation of film distribution activities 
at the Library’s National Nfcdical Au- 
diovisual Center in Atlanta. A computer- 
ized system is being developed for Wak- 
ing films, preparation of shipping docu- 
ments, inventory and suspense control, 
collection of utilization data, and reports 
to program sponsors. Plans call for a sys- 
tem vvmich would combine source data 
automation and data processing tech- 
niques for maximum manipulation of 
data to produce the necessary reports, 
and for handling loan transactions with 
maximum speed and efficiency. 

There is significant developmental 
work being conducted in other medical 
libraries to apply the new computer 
technology. The organization cf a new 
medical school library at the Urn versify 
cf New Mexico has offered a unique op- 
portimily to devdop machine methods 
cf bibliographic control and to use elec- 
tronic data processing for library func- 
tions. The University has been the recip- 
ient of a research grant from the Nation- 
al Library of Mcoicine to develop a to- 
tal system of library operation based 
upon such aulomatca support. The dc- 
vxlopmcnt program has included the 
functions of acquisitions, cataloging, se- 
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rials control, accounting, circulation and 
inlerlibrary loan. The serials control pro- 
gram has sho'svn successful use of EAM 
equipment for the past four years for 
the activities of ordering, receiving and 
processing, claiming^roducing current 
listings, and others. The circulation pro- 
gram includes the use of the IBM 357 
Data Collection System for control pur- 
poses. A borrower presents a pre- 
punched plastic identification card and 
this with a prepunched book card gener- 
ate a transaction record. This fs reported 
to be an effective, accurate, fast and rel- 
atively simple process. Preparation of 
overdue notices is done in a matter of 
minutes and management information 
about circulated items, borrowers, etc., 
is made readily available. 

Current work is being conducted on 
adapting to impro\*cd machine capabili- 
ties at the Washington University 
School of Medicine Library, St. Louis, a 
leader in library automation. (14) In 
19B3, this library was reporting efforts 
In the area of automation, specifically 
<hc printing of acquisition lists from 
magnetic tape files and the publishing 
of periodical holdings first from cards 
and later from tapes, A year later re- 
ports were releases on the operational 
circulation rciords and control system 
and more recently this library has de- 
scribed its sj'slcm for producing book 
catalogs from a single punched card in- 
put, manipulated by a computer under 
the control of programs written for the 
task. Work continues to improve these 
operations and to expand the support re- 
ceived from automation. For example, 
the acquisitions and cataloging records 
stored on magnetic tape ha\Te been ex- 
amined to determine frequency and av- 
erage length of record components and 
their alphabeti/at'On requirements In a 
l)ook catalog. Data from this study are 
being used in adaptation of the library*! 
computer-based cataloging sjitem to 



greater machine capabilities. 

The Slate Universities of New York 
(SUNY) have undertaken a series of 
projects in the various areas of library 
automation. A principal effort was the 
use of a computer to support the publi- 
cation in 1965 of a State University Un- 
ion List of Serials intended to facilitate 
the sharing of serial resources. The first 
edition was published In the fall of 1966 
and contained 17,000 titles; the second 
edition, dated December 1967, included 
25,000 titles. This publication has trig- 
gered a significant increase in inlercam- 
pus exchange — or interlibrary loan. The 
intcrunivers ity transaction at the Up- 
state hfcdical Center rose from almost 
zero to 1S20^ of the total activity at that 
library, SUNY has also assumed a role of 
leadership in the development of a 
biomedical communications ret%vork, 
described later in this report. 

The medical libraries of Columbia, 
Harvard and Yale collaborated on a 
computerization project including the 
printing of catalog cards and biblio- 
graphic lists. Their first product w’as a 
monthly accessions list that appeared in 
October 1963. Although this project has 
since been terminated, it represented a 
significant effort In cooperative library 
actirity based on computer support. The 
Yale University Medical Library, contin- 
uing with a computerization project, de- 
veloped a program in 1966 to produce 
book-form catalogs arid catalog cards, 
using the IBM 7091-7040 Direct Cou- 
pled System. An acquisitions and in- 
process control system utilizing comput- 
er support has also been designed by 
Yale covering the acliritics from request 
for purchase to cataloging and shelring. 

In/ofinofion Sysfem Erolutfflon 

Arc our efforts to automate accom- 
plishing the interded purpose? Have w*c 
jpent money for apparent sophistication 
but not really improved In the efficiency 
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of our job, the quality of our work or, 
the most important criterion, the service 
to Our customer? In many cases, the 
only honest answer is we don’t really 
know. The capability to evaluate the re- 
sults of our efforts scientifically has 
proved to be a very difficult task. As 
itulh Davis, Associate Director of Re- 
search and Development at NLXf, has 
stated . . . “The nopcs or wishes of 
those concerned with the evaluation of 
information systems is to apply methods 
of the physical sciences to such evalua- 
tions.” (15) She goes on to point out the 
fallacies of such approaches for systems 
where the human being — both as an 
obser%’er and user -* play's such a domi- 
nant role. Observation itself can effect 
change. The user of the information sys- 
tem Is often considered a part of the sys- 
tem itself and yet represents en area cf 
many unanswered questions. On the 
other hand, John Tukey of Princeton ex- 
pressed the opinion that there arc, in the 
information systems area, things to do 
that pjc obviously useful enough to need 
no formal detailed testing. He has con- 
jectiircd that **lhe real proof of the value 
cf change is how muen it hurls If you 
try to return to the old ways," ( 15) 

Ihere have been a limited number of 
cases where major operational informa- 
tion systems have been subjected to 
large-scale evaluation efforts. A national 
conference on Electronic Information 
Handling was held in Pittsburgh fn 
April of 1967. It was designed to focus 
Attention on new ideas, research and dc- 
VTlopmenl In the techniques of testing 
and evaluating electronic information 
handling systems. The report of the 
conference is contained in the publica- 
tion titled Eltefronic Handling of /n/or- 
matloni TesUng and PAohitition, edited 
by Kent, Taulbce and Brisker. (15) It 1$ 
recommended as an excellent reference 
on this difficult subject. In addition, a 
chapter hai been dexoted to the subject 



of Evaluation of Information Systenis 
and Services and ^vrilten by Alan Rees 
in the Annual Review of Information 
Science and Technology, Volume 2. 
(16) This covers a general discussion of 
testing and evaluation as well as a re- 
view of the efforts in laboratory-based 
testing and evaluation, tests of operating 
systems, and research in problem areas. 

One of the few major information sys- 
tems to “bare its sour to the public is 
the MEDLARS system. The evaluation 
of this system is the most elaborate and 
sophisticated performance study to date 
of an operational system. A complete re- 
port of this study by Lancaster is now 
available through the Government Print- 
ing Office. (17) 

The study concentrated cn the evalu- 
ation of the demand or retrospective 
search function. Over 5,000 searches per 
year arc performed at NLM with addi- 
tional searches being run at deccnlral- 
tzed centers In the United Slates and In 
England and Sweden. The principal ob- 
jectives of the lest wcrci 1 ) to study the 
demand search requirements of the 
MEDLARS users; 2) to determine how 
effectively and effrcicnlly these require- 
ments are being met; 3) to recognize 
factors adversely affecting the system; 
and 4) to disclose ways improvements 
can be made. The evaluation was a com- 
plete system evaluation since it Inchidod 
all components affecting the perform- 
ance of the system as measurco by the 
satisfaction of the MEDLARS users. 
The study was guided and reviewed by 
an external board of advisers, who wore 
selected for their prc-cminenco in the 
fields of Information science And/or sta- 
tistics. 

The analysis of a carefully selected 
sample of 300 searches revealed that for 
an "average" search somewhat over 
onc-half of the citations fn the file that 
were relevant to the request were re- 
Iriotd and that of those citations that 
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were retrieved between one-half and 
two-thirds were judged to be relevant 
by the requestor. These are coasidered 
to be good results when compared with 
the performance of other systems, al- 
though such results are difficult to com- 
pare' Iwause of the variety of methods 
used to arrive at the expressions of per- 
formance. 

A principal cause of the failure to re- 
trieve a relevant item w^as that the in- 
dexing was not cxhauslix’C enough. Re- 
trieval of inelcvant items was due, at 
least in part, to lack of specificity in 
searching. However, the greatest poten- 
tial for improvement was judged w 
the interface between the user and the 
system. Over m of the system fillurcs 
were traced to defective interaction in 
this area. There is need for more intel- 
lectual effort and care in the statement 
of the requirements on the part of Ac 
user. Other results highlighted Include: 
1) need for improved vocabulary con- 
trol and updating; 2) 453P of Input effort 
required for foreign lan 55 uagc material 
with comparatively small acceptance of 
users; 3) bibliographic record as output 
to the user is not adequate to indicate 
atticle content; and 4/ the need for a 
continviing quality control mechanism 
for the future. These areas arc all under 
present consideration by the manage- 
ment of NLM. 

Wetuorking end Cooperatkx ActMiks 

There is little evidence In the litera- 
ture that there has been a change from 
the absence of a ‘'total system’’ con^pt 
of library automation as noted bv Black 
and Farley in their re\*icw of the 1965 
literature, (8) They expressed the opin- 
ion that . . . *Wcrc a total s>-stem con- 
cept ever to l>3 implemented within a li- 
brary, U wxjuM certainly blur the dis- 
tinctions lielween such traditional dc- 
parlments” . . . (circulation, acquisi- 



tion, cataloging, serials, etc.). They con- 
tinued . . . "This leads one to suspect 
that much of the reluctance on the part 
of many librarians to undertake work in 
automation may stem from the fear that 
automation will force significant 
changes in exisling organizational stiuc- 
tures. That fear is certainly justifi^. 
However, the continued fragmentation 
of library operations tends to isolate the 
operating divisions from their goal, 
which is, of course, service to a group of 
users." 

It is recognized as difficult to make 
radical changes in large operations but 
the real advantages to oe offered by au- 
tomation will not be realized until the 
entire library is recognized as a single 
system and treated accordingly. In reali- 
ty, this is not even a broad enough out- 
look, The individual medical library 
should be considered as a subsystem, or 
component, of a much larger system — 
the national biomedical oommuntcations 
network. AVc believe that the local H- 
brar, can be developed Into a respon- 
sive biomedical communication resource 
as an integrated part of an expanded 
comprehensive sendee network. A 
stronger national network is urgently 
needed to serve the infonr»aUon needs of 
all personnel in the medical sciences. Us 
aim should be to permit equal and rapid 
access to all typos of bl^jmcdical infor- 
mation. 

Networks to librarians arc nothing 
new. Well before this word became 
fashionable and before the advcn^ cf 
modem computers and electronic com- 
munication systems, interlibrary loan 
networks among libraries on a regional 
and national scale were created and op- 
erated by members of the profession. 
Tl^e complex systems of national aiKl re- 
gional bibliographic control in the form 
of union lists and catalogs, of interim 
source rcfcnal senices, and the large 
and growing traffic in inlerlibrary loans 
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clearly identify the library system as a 
viable de jacto network, not ordinarily 
recognized as such by its patrons. But 
with the newer tools provided by our 
advancing technology, the netw^ork 
takes on a completely new dimension. 

The planning, desim and implemen- 
tation of the Biomedical Communica- 
tions Network has become a principal 
task of the new research and dev'elop- 
ment office at the National Library of 
Medicine. This Network is the embodi- 
ment of NLkis plans to provide im- 
proved access to information within the 
medical community for the support of 
continuing education, individual practi- 
tioners and researchers, and institutions 
such as hospitals, clinics, and research 
centers. 

At least for initial planning purposes, 
the Network has been divided into sev- 
eral components by l)pc of service to be 
rovided. These components are for li- 
rary scnices, specialized information 
senices, specialized educational services 
and audio and audio-visual services. 
Supporting these four service areas will 
be the data processing centers and 
transmission facilities. 

Planning Is most advanced for the li- 
brary component, as might be expected. 
Prcaating this formalized program was 
the establishment of decentralized cen- 
ters at UCLA, Hanard, University of 
Michigan, University of Alabama, Uni- 
versity of Colorado, and Ohio State for 
MEDLARS searches. Search formula- 
tion Is accomplished at all of these cen- 
ters, with cards punched for computer 
input. Some of the centers arc actually 
running their own machine searches us- 
ing MEDLARS computer tapes provid- 
ed by NLM and some are sending the 
punened cards for searches to be made 
on NLM's equipment. Several arc now 
transmitting the search requests to NLNf 
through card readers and telephone 
communications linb, thus, creating a 



duplicate card deck at NLM for input to 
its computer. 

An early pari of the research and de- 
velopment program has been the estab- 
lishment of a Remote Information Sys- 
tems Center to serve both the library 
and specialized information components 
of the Network. This Center is the ter- 
minal facility at the Library for commu- 
nications links to a variety of data sys- 
tems and provides remote access to sev- 
eral computer centers and to the decen- 
tralized NiEDT^RS centers. The pur- 
poses of these links are to provide access 
to remote data banks, support experi- 
mentation with the research and dev'cl- 
opment of others in the information and 
computer sciences, provide a capability 
to demonstrate techniques and the 
state-of-the-art In information storage 
and rcUieval, and to assist in the data 
manipulation needs of the Library. The 
ability to sit at a telct)*pc in the Center 
and to query a file of citations stored on 
a computer in California, to receive al- 
most instantaneous resnonsc with the ci- 
tation requested t)^^^ in front of you 
by data transmission from the computer 
is truly a remarkable experience. And 
yet Ibis is unsophisticated input/output 
compared with the cathode ray tube 
devices now available. Such remote 
query capability utilizing computer time- 
sharing will be a principal clement of the 
planned services and, hopefully, in the 
not-loo-divlant future. Tlic cst.iblishmcnl 
of this center represents the first tangible 
manifestation of communications net- 
work activity at the Librar)*, It will serve 
as an operating Input/outpel point for 
the library and specialized intormalion 
service components of the Network. 

The planning for the specialized edu- 
cational services pvlaccs an emphasis on 
the needs of continuing medical educa- 
tion but includes the needs of the medi- 
cally uninformed, particularly those 
without the knowledge of basic and es- 
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sential family health rules. Television is 
the principal medium being considered 
and plans include the use of satellites to 
aid in transmission. This is a radical de* 
parture from the traditional activity of a 
library, and is an example of a need that 
has l^n expressed for the applications 
of new technology to the medical infor- 
mation requirements of sodety. 

Biomedical communications network 
planning and experimentation is also un- 
derway in the State Universities of New 
York (SUNY). In 1965, the SUNY ad- 
ministration received a proposal from a 
task force recommending (lie establish- 
ment of a biomedical communications 
network. This network was to link the 
Health Sciences Library at Buffalo, the 
Upstate Nfcdical Center at Syracuse and 
the Downstate Medical Center at Brook- 
lyn. It WAS proposed as a computer-cen- 
tered neb^'ork and recommended to 
scr>’c as a prototype for the University- 
\nde library system which includes M 
separate libraries. The stated purpose 
was to retrieve bibliographic informa- 
tion relating to books and journals 
owned by the three libraries ana by the 
National Library of Mcdidr;e. Later 
plans called for the addition of material 
in related sciences and* finally, for auto- 
mation of In-house routines of the librar- 
ies, leading to cooperative or coordinat- 
ed acquisition and processing of new 
maleiials. SUNY accepted the proposal 
and designated it as a pilot study. The 
network Is to begin ope rations in the fall 
of 196S. 

Network development and plans 
ur>dcrway in soxral other stale libr<r) 
groups. For example, medical librarians 
nave recently organized the Texas 
Council of Health Science Libraries to 
Improve their cooperathT, tnlcrlibrary 
activities, and they are looking ultimate- 
ly to some type of network Interconnec- 
tion. Teletype senices are already being 
used to support Inlefltbrary loan activl- 



ti(‘S among five of the institutions In the 
North Texas Interuniversity Council, 
and in the Gulf Coast area a Regional 
Information and Communication Ex- 
change centering on Rice is experiment- 
ing with teletype and computer services 
to participating libraries in all of the 
coastal counties. 

Cooperative actions among libraries 
of the United Slates have loo frequently 
been limited to interlibrary loaning of 
materials, discussions at conventions, 
and discussions in the literature of com- 
mon operating problems. Automation 
has increased tne awareness of the need 
for improved and extended cooperation 
because of the potential advantages of 
sliaring work and resources through ex- 
change of records and linking by net- 
works. A new ora In library cooperation 
w^as opened in June 1967 when the di- 
rectors of the tluce national libraries — 
the Library cf Congress, the National 
A^cultura! Library and the National 
Llorary of Medicine — announced the 
establishment of Ore National Libraries 
Task Force on Automation and Other 
(kx)perativc Services. The purpose — to 
spcc^ the flow of research information 
to the nation's libi uics and to the schol- 
rtrf and researchers who use them. 

The Task Force has issued its first 
progress report which contains a list of 
desirable products and services to be dc- 
iiv<.d from the con^bined efforts of the 
three ! t b r a M e s, fncluding book 
catalogs, machine-readable cataloging 
data, demand bibliographies and coordi- 
nation of classification, acquisitions and 
reference services. The report’s rccom- 
nendvlions arc now being used as a ba- 
sis for further disaissions. Working 
gioups have been established to altaek 
each problem or functional arcA arn! 
there are signs of real progress toward 
unified action. 

Special mention should be made of 
the progress lovvaid the establishment of 
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a standard format for machine-readable 
cataloring (MARC) and the distribu- 
tion of catalog tapes In that format by 
the Library of Congress. Much of this 
work on MARC was accomplished by 
the Library of Congress prior to the es- 
tablishment of the Task Force but now 
is included in its program. Details of the 
project are covered in the reports au- 
thored by Henrieltc Avram and others 
listed in the references. ( 18) ( 19) 

Representatives of the three libraries 
arc also meeting to discuss and plan for 
a system framework srilhin wnich to 
place the total effort It is hoped that 
not only s\ill this work lead to a forerun- 
ner of a truly national library network 
but that it will stimulate others in the ll- 
biary community to increase their coop- 
erative efforts. 

Somnwiry 

The use of computers in libraries can 
be summarized as supporting two main 
purposes: the handling of records gener- 
ated in the library s operation, and the 
automatic extraction and codification of 
information in the library s files for use 
by clients. The feasibility, utility, or 
technical availability of the former ap- 
plication is well documented. The latter 
task requires considerably more research 
1 ) 0 fore practical application uill be com- 
mon. 

The emphasis that has been placed on 
automation, computers and the advances 
in Information science should not 
leave the reader with false Impressions 
concerning the future of the medical li- 
brar)’. In spite of the impressive changes 
in the technology of information storage 
and retrieval, the traditional library >\ill 
continue to serve as a sital instrument In 
the educational process. 

fn the report prepared by the Library 
Study Committee of the Assodalion of 



American 'ledical Colleges titled the 
Hechh Sciences Library: !(s Role in Ed~ 
ucction for the Uedth Professions, this 
position was supported by a listing of 
several cogent reasons. (20) Included 
were the need for the display of the 
printed word through traditional meth- 
ods, the requirement for the housing of 
certain documents that are classics or 
original in their presentation, and the 
need for the older portion of the scien- 
tific record that uill probably not be 
economical to transfer to some newer 
media. 

This is not to end, however, on a note 
of status quo for the libraries. They 
must begin an active pcrivxl of rcscarcn 
into the most effective ways to dissemi- 
nate material that is of paitiailar inter- 
est to the individual scientist and mate- 
rial that is new. The libraries must he 
shilling to accept an educational role 
that is much beyond where they stand 
today. This should include at least the 
education of the student of the health 
sciences with the form In which \he 
scholarly record is contained and the 
means oy which one gains access to it, 
and to identify, attract, and educate 
new talent for the Increasingly special- 
ized field of information management. 
Finally, the health science library will 
have to adopt a new philosophy in the 
area of service. It must Include eventu- 
ally the provision of linkages between 
people and information as well j be- 
tween people and documents^ availabili- 
ty to the latest significant current events 
in professional societies, conventions, 
meetings, hospitals and laboratories; and 
development cf selective dissemination 
sem'ces. The challenge is before the li- 
braries and the scientific and technical 
community is demanding that they ac- 
cept It. 
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SUMMARY 

CX)MPU1'ERS IN MEDICAL LIBRARIES* 

Panel: Martin Sf, Gumming** Nf.D., Chainnan, Irwin Pizer, Robert T. Divett, Leonard Eddy. 



The pane! on Compuleis in Medical 
Libraries was pleasantly surprised to 
find an audience of approximately 30 
participants, and tlie word “partici- 
pants'* should be emphasized l^ause 
more than half of the alloted time was 
spent in active dialogue between partici- 
pants and panel members. In an attempt 
to meet the charge, a statement of gen^ 
era) purpose has been developed. 

The importance of the medical library 
as a learning resource for education Ls 
reflected in many way’s. Local, state and 
federal institutions an i organIzatio;is 
are deploying more fun<ls and manpow'- 
er in an effort to restore this iristrumenl 
of education and learning to a tool of ef- 
fectiveness and efficiency geared to 
match the expansion of the total educa- 
tional needs of our nation. Libraries are 
seizing this opportunity to use the com- 
puter and related new technologies to 
solve some of the existing Information 
problems. There Is still need, however, 
tor refinement and further discrimina- 
tion in application of computers to this 
enormous set of communications prob- 
lems. The most encouraging cvioencc 
that success is close at hand comes from 
conferences such as this one in which 
educators, administrators and librarians 
are sharing vnews, experiences and rela- 
tionships critically, candidly and cooper- 
ali\e)y. To share resources on a national 
basis, plans for doTloping local, region- 
al and national library net>^orks seem to 
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be advancing at a rapid pace. The need 
to consider standards for compatibility 
among computer systems has been rec- 
ognized and much work is uadenvay to 
bridge program and hardware dispari- 
ties in an effort to effect economy and 
efficiency of operations, 
ft has been estimated that more than 
OCWI of all health personnel that have ever 
dedicated their lives to research are 
alive and working in laboratories and 
clinics today. As a result, the immense 
growth of knowledge and printed matter 
since World War II has caused a great 
concern among those who need to find 
and use this information. In 1963, it was 
estimated that the growth rale of publi- 
cations over the prevnous 20 years had 
been about 3.1? per year, doubling every 
22 years. More recent estimates reflect a 
doubling every 1 1 years, ft was further 
estimate that information In the world's 
libraries is growing at a rate of 2 x 10* 
bits per second (a bit in this context 
being approximately equivalent to half 
of one feller or one numeral). 'This 
grouih has created 'evere strains that 
arc beginnii^z to b ‘itk down the effi- 
ciency of all methods of information 
handling that heretofore have been rea- 
sonably adequate. As a result, a new 
methodology of information handling is 
developing. 'This methodology is based 
upon the ability of computers to per- 
form some of the routine functions of In- 
formation management and processing 
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previously handled by man alone. The 
predecessor of the computer was the 
punch- card accounting machine devel- 
oped by Hollorilh ana Billings, and the 
use of the punch -card equipment in li- 
braries goes back to the early 1930 s. 
Since that time, a few libraries have 
been using this methodology for some of 
their routine library procedures. Among 
the first libraries to use punch-cards was 
the University of Texas whore Ralph 
Parker developed a punch-card circula- 
tion system, since adopted by many oth- 
er places. 

There have been two schools of 
thought concerning the use of comput- 
ers in information handling. The librari- 
an, for the most part, has looked at the 
computer simply as a moans of accom- 
plishing the traditional tasks of libra- 
rianshjp. The documcnlalist, on the oth- 
er hand, has looked at the computer as a 
potential means of storing, rctrie\*ing 
and manipulating information of all 
types. Vanncvar Bush in 1^15 published 
tne provocative paper which has led to 
the most revolutionized thinking of 
those who are concerned with informa- 
tion management. From h!s report has 
germinated the concept of fact retrieval 
as opposed to citation retrieval, which 
has alw^*s been the librarian’s ap- 
proach. The ability to retrieve citations 
is a necessary prelude, however, to ariy 
effective machine retrieval of actual 
fact. Except in a scry few cases, the 
technology of information science has 
not yet cfcvclopcd to the point whore 
there can be fact retrieval oy machine. 
The National Library of Nfcdicinc has 
had some success witn its medical litera- 
ture anaijsts and retrieval system, but 
its principal contribution has ocen to in- 
trOQuee computer technology on a fairly 
extensive scale within the library set- 
ting- ll Is now planning the develop- 
ment of a second MEDI^RS system 
which will allo^v the use of time-snared 



computers in retrieving not only refer- 
ences or citations, but abstracts and se- 
lected portions of total text as w’ell 
Serious investigation of the use of 
computers for all library operations be- 
gan W'ilh the development of the new 
campus of the University of Illinois in 
Chicago. Shortly thereafter, one of the 
leaders of the team at Illinois accepted 
the position as Director of Library and 
Information Services at the new Florida 
Atlantic University in Boca Raton. Dr. 
Heiligcr put into effect at Florida Atlan- 
tic the plans preWously developed at the 
Chicago undergraduate campus at the 
University of Illinois. I think \vc should 
point out that the Florida Atlantic pro- 
[cct was not an unqualified success. Tn 
fact, some would say it was an abysmal 
failure and the panel bt lieves that refer- 
ence to failures is important to recall 
and that we should not overemphasize 
our successes. Shortly after the Illinois 
report Dr. Estelle Brodman and Mr. Ir- 
win Pizer began a project aimed at total 
system library operations at Washington 
University School of Medicine in St. 
I.ouis. 

Selected library operations In this 
project have been developed successful- 
ly using the batch mode of computer op- 
erations. About the same time, Fred Kil* 
our. Librarian at the Vile Nfcdical Ll- 
raiy, conceived of a project that would 
place the holdings of his great Library, 
those of the Harvard Medical School Li- 
brary, and those of Columbia University 
Nfccfical Library onto a central comput- 
er storage deuce connected to each P- 
brar)' by leased lines and tj^^K writer ter- 
minals using time-sharing principles. Al- 
though Kllgour had apparently worked 
earruilly and meticulously to develop 
the softw are for this program, this effort 
failed for an entirely tlifferenl reason. 
Harvard wvlhdrcw from this project, 
presumably for administrative or organi- 
zational reasons, so there is a sccomfeM- 
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dence of failure, but in this case appar- 
ently not caused by a lack of technical 
planning. 

One cannot attempt to reviev/ devel- 
opments in this field without making 
reference to the exciting work which has 
been taking place at the Massachusetts 
Institute of Technology during the past 
decade and, in the interest of time, will 
only refer to the aocomplishmenls of 
project MAC and its related and subsid- 
iary project TIP wMcb has demonstrat- 
ed the ability of vsers to work on-line 
with massive dais files generated, In 
fact, by the users ihcmscKes. The con- 
cepts emerging from this experiment 
have been seized uoon by other groups 
in the evolution of s >bscqucnl on-line li- 
brary and data-based information devel- 
opments and perhaps the most signifi- 
cant of these is the second Nf J.T. proj- 
ect, INTREX, which vnll attempt, bc- 
^vecn now and 1975, to develop a fact 
retrieval library'-basod Information sys- 
tern within the field of interest of cn^- 
ncering and other hard sciences. 

Returning to iricdicinc, in 1963 the 
University of New Mexico began to es- 
tablish their new school. The University 
of New Mexico School of Medicine Is a 
relatively young school, graduating Its 
first class in June, 1968. The School of 
.Medicine Library was faced with the 
task of building an adequate library in a 
short lime. It was early decided to uti- 
lize machine methods to the fullest ex- 
tent possible. The first few months w*cre 
spent in designing a punch-card system 
and in putting records in machinable 
form. It was found that no total s)'slcm 
was available for use in the Library. It 
was necessary to borrow, modify, and 
redesign several programs to arrive at 
usable solutions to problems encoun- 
tered. This punch-card system has been 
gradually phased out in the past 
months, and is being replaced by batch 



processing on on IBM 360 Model 40 
Computer. 

New Mexico is now doing research to 
design a total library on-iinc system — 
one w'hich can, with only slight modifi- 
cation, be used in other libraries. One 
requirement is that these programs be 
WTilten as nearly as possible in a univer- 
sal language. An IBM 357 Data Collec- 
tion System is now being utilized for cir- 
culation procedures, and a daily print- 
out Is obtained, much as is done in many 
libraries across the country. 

Planning and implementing of on-line 
procedures is w^ell under way. Two IBM 
220 CRT Terminals will be installed in 
the Library in June of this year, to be 
connected to the IBM 360 Computer. 
These terminals will be utilized in ac- 
quisitions and cataloging processes, as 
well as for serials control and circulation 
procedures. 

Acquisitions and cataloging programs 
will DC closely interrelated when the 
CRT Terminals arc In operation. On re- 
ceipt of a book request, the acquisitions 
librarian will place Ibis rccorrt on the 
computer. From this lime until final ca- 
taloging, the original record will be re- 
called and supplemented or ccrreclcd as 
necessary on tnc CRT Terminal. 

The serials control program Is similar 
to acquisitions procedures. A descriptive 
catalog entry will be made for each se- 
rial title. Receipt of a serial will be re- 
corded on the CRT much as Is now 
done in a Kardex file. Claiming proce- 
dures arc moie difficult, 1 he vagaries of 
publication and distribution ot serials 
make reeessary the development of a 
man-machine interface to effectively ac- 
complish this necessary claiming opera- 
tion, There is an anticipation period for 
each serial, a time period within w hich a 
pwliailar issue is expected to arrive. 
This period might ho 30, 60, or 90 da)'S, 
for instance. Should the item not bo re- 
ceisxd during the anticipation period, 



this fact will be brought to the attention 
of the serials librarian. She will decide 
whether to claim any particular serial, 
and a completed claim will be produced 
when required. 

The primary impact on tlie user of the 
Library will be that, Instead of going to 
a card catalog, he w^ill go to a cathode 
ray tube terminal to begin a literature 
search. The computer will be pro- 
grammed to ask questions of the user. 
These questions and his answers will, 
hopefully, duplicate the thought proc- 
esses which t^c place in us3 of a tradi- 
tional catalog. The user should be able 
to locate needed information in less than 
a minute. Appropriate citations will then 
be displayed on the terminal or printed 
out on an attached printing unit as de- 
sired. One additional search method will 
be possible with the software being de- 
veloped — a TWX search. Any teletype 
station should be able to dial into Inc 
computer system and do a literature 
search to any depth. 

For all practical ptirposes, the com- 
puter system and library being devel- 
oped by the University of New Mexico 
School of Medicine is not a local system, 
but rather 1$ a statewide or even region- 
al information center. 

This model was discussed in some 
depth by the panel with a Wcw of deter- 
mining whether a system developed at 
one new medical school might be trans- 
lated or transferred to other new medi- 
cal schools. 

The panel then spent much lime re- 
viewing ihc interesting dcvclopmcr^ts on 
a state-widc medical library network 
which is being developed by the Stale 
Univ'crsity of New York. 

The Stale University of New’ York 
Biomedical Communications Network 
began in 1965. The Nctw-ork now links 
Slate University Nfedical Center Librar- 
ies at Buffalo, Spacuse, Brookl)m and 
Stony Brook. In addition, the Unhersity 



of Rochester Medical Center and Alba- 
ny Medical Center Libraries have joined 
the Network. The collections of these li- 
braries total more than 600,000 volumes 
and more than 9,000 serials are currently 
received. 

An IBM 360 Model 40 Computer with 
IBM 2740 Communication Terminals 
are utilized in the Netw’ork. Exchange of 
interlibrary loan data, reference requests 
and other administrative messages con- 
cerned with the libraries and the Net- 
w*ork are facililatcd as secondary appli- 
cations of the Nchvork. 

The Network is primarily orienled to 
the user wnth goals and functions dearly 
slated as follows; 

1. Production of a computerized catalog 
of monographs in the three SUNY 
medical libraries, published from 
1962 onw ard. 

2. Production of union lists of serials 
and books wnth periodic updating 
(this may include the medical librar- 
ies, the entire 5S campus SUNY Li- 
brary System, or a combination of li- 
braries w ithin the system). 

3. Search of the comoined catalogs for 
alt libraries for monographs bearing 
iniprinl dates of 1962+- Eventually 
10 years of book data will be stored 
on-line. The initial data base consists 
of approximately 20,000 records. 
These can be searched by author, ti- 
tle, subject, and other special fields if 
desired. 

4. Search of 5 years of MEDLARS data, 
to be obtained from the National Li- 
brary of Nfcdicine. This amounts to 
approximately 1,000,000 records, also 
stored on-line. These will be able to 
be searched by author (name) or 
subject, with the usual limiting 

ara meters of lime, language, scjt, 
ale, etc. 

5. Search of the SUSY Vnhn LM of 
Scriah data consisting of over 20,000 
cunent and non-current periodical titles 
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held in the 60 libraries of the State Uni- 
versity, plus Syracuse University, Hamil- 
ton College, Colgate University, and 
several smaller schools and special li- 
braries. 

6. Preparation of lists of cuncntly re- 
ceived titles for each campus, or the en- 
tire system, of books — arranged by au- 
thor or subject, and of journals — ar- 
ranged by title or subject, or any desired 
combination. The inform'ttion dissemi- 
nation aspect of this phase is of great 
im^rtance to all faculty, research staff, 
ana student users, 

7. Education and training of library 
personnel and orientation and training 
for users. 

8. Current awareness scrNiccs — Selec- 
tive Dissemination of Information 
(SDI). Individual faculty members or 
research groups could be notified of 
new articles and books in their special- 
ized fields of interest as each new 
M EDITHS tape is received and as cata- 
log information is added to the master 
files. 

9. Key Word in Context (KWIC) in- 
dexing of appropriate local research 
d ta, and. if m^slred, of Journals not cov- 
ered by the major indexing services. 

10. Recurring bibliographies, c.g,, 
monthly for a specific research group or 
projcci. 

11. Research and development of 
techniques for information storage and 
retrieval (see below). 

The major elements of the data base 
will be augmented by the circulation 
fc-cordf for the Upstate Nfedieal Center 
Library and the other libraries as they 
develop autom;.'tcd circulation s)-stcms. 

The user will query the svstem him- 
self, not through kn intcrmcoiar)*, in his 
own terms, which will be translated, if 
necessary, inlcrnallv. Within two min- 
utes of the completion of his query 
statement he will begin receiving the re- 
spoa^es. 



NLM Shared Cataloging Program 
In 1966/67 the National Library of 
Medicine negotiated agreements to be- 
gin expanding the scope of the Current 
Catalog, As stated in the 1967 fiscal year 
Annual Report of NLM ; 

Negotiations were completed for a 
manually shared cataloging project 
w^hich will result in the Francis A. 
Counlway Library’s acquisitions being 
Included in the CURRENT CATALOG. 
In FY 1968, the Francis A. Counlway 
Library will begin submitting master 
forms of their catalog citations for inclu- 
sion in the CURRENT CATALOG. 
Those which are added to the catalog 
will reflect the NLM classification and 
MESH subject headings, as well as the 
Counlway Library sy^nwl, MBCo. 

In addition, 

SUNY/NLM Shared Cataloging Proj- 
ect arrangements were completed witn 
the SUNY Upstate Medical Center, Syr- 
acuse, New York, to: 

A. Convert NLM current cataloging 
tapes to IBM 360/40 and to refor- 
mat the NLM data according to the 
current tape formats to project 
MARC of the Library of Congress. 

B. Provide programs for searching of 
the resultant tapes using IBM's CIS 
document handling module. 

C. Experiment w*ith batch searches of 
the resultant tapes, 

O. Share input in machine cataloging 
data. 

E. Cooperate in designing a shared ca- 
taloging S)-stcm. 

Beginning in April, 1968 both Count- 
way and SUNY items will appear in the 
Current Catalog, and the contract be- 
tween SUNY and NLM has been re- 
newed in order to compile and search a 
master data base consisting of NLM, 
Counlway, ami SUNY machine readable 
records. 

A total of 32 full-time persons arc cur- 
rently engaged in the implementation of 
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the Network, In addition to the existing 
staffs of the component )ibraries who 
number 92 and who may be thought of 
as resource persons for the NelworJc pro- 
gram. 

Paralleling this dcs'clopment of 
mechanization ot IradilionaT library 
procedures has been a significant 
change in the concept of Ubrarlanship, 
especially medical librarianship. The li- 
brary function has been defined as the 
conscious Interruption of the total 
stream or flow of recorded Ideas and In- 
formation, the selective drawing off of 
manageable amounts for storage, later 
retrieval and distribution to individuals 
or groups in whatever media forms or 
formats might be rcrjulrcd and arc ap- 
propriate to satisfy known or anticipate 
needs. In other words, libraries will be 
increasingly concerned with manage- 
ment of information In whatever format 
it is in or the media used to disseminate 
It. 

In response to the question 
will libraries look like in the future?", 
Stone has slated the library of the ftiturc 
is not wisci) conceived as a single place 
at ail, but rather as a far-flung network 
composed of units of various sizes and 
shapes, each of which may perform sim- 
ilar as well as different functions, but all 
of which will be linked together elec- 
tronically or mechanically. In the fiiturc, 
it will probably bo less nccess.iry to 
have all of the pieces of a librar)* pro- 
gram in one place so long as the pro- 
gram parts can be linked together. In 
other words the concept has developed 
that the library should bo a place from 
user, lather than a place where the user 
which Information is disseminated to the 
must come to acquire the Information. 

The Ircrxl seems to bo devTlopIng, 
particularly In newer medical schools, 
toward the centralization of administra- 
tion of all agencies concerned with the 
management and communication of in- 



formation. Among those places where 
computers, medical illustration activi- 
ties, educational media development 
and library science operations are con- 
solidated under one administrative head 
are the University of New Mexico 
Sciiool of Medicine, the University of 
Texas Medical Schools in Dallas and 
San Antonio, the California College of 
Medicine of the University of California, 
the University of Arizona School ot 
Medicine and more recently, the Univer- 
sity of Alabama Health Center. In those 
cases where librarians arc qualified to 
serve as Information sclenlisls, the li- 
brarians have been placed In charge of 
this administrative unit. In other places, 
men from science or medicine who have 
become interested in the modalities of 
information transfer are assuming this 
Important role. 

The consolidated operation lends It- 
self toward the development of the tech- 
niques of information handling that are 
now on the horizon, but not sufficiently 
developed for implementation for five or 
ten years in the lulurc. Among these are 
the methodologies of fact retrieval ear- 
lier referred to. Such techniques may 
use microfilm, microform, video tape, 
motion pictures, facsimile transmission 
and oilier media. If there is no central 
managemenb implementation and dcvel- 
oprijcnl of medical information systems 
will be hampered and, indeed, might bo 
chaotic in terms of potential linkages. 

At the national level, the National Li- 
brary’ of Medicine has underway a mas- 
sive systems dcsi^ for biomedical com- 
munications whi^ Involves several ma- 
for information components. They arc 
(1 ) an automated document handling li- 
brary network, (2) the development of 
a network in\x>lving the National Medi- 
cal Audiovisual Center and its concern 
with other learning resources, (3) sys- 
tems management through monitoring, 
control fccdoack and application of con- 



trol system procedures, (4) evaluation 
of systems operations so as to extract 
systems parameters, (5) systems inade- 
quacies and component coupling, and 
(6) the support of education through 
automated Information and communica- 
tion networks, including the possible use 
of satellites for public as we!! as for 
medical educalion, AH of these efforts 
are related to tstabb'shing compatibility 
among the three national libraries, as 
well as with specialized information 
centers and other emerging national in- 
formation programs. Medicine presently 
is in the forefront of t^iCsc efforts and in- 
tends to share fully its experience \\*ith 
other disciplines of science and technol- 
ogy, The methods developed by NLM 
for decentralizing its computer-based In- 
formation programs and, particularly. Its 
extension into all levels of support for 
medical research, education and prac- 
tice has led the President to include it as 
a prototype for the developing “knowl- 
edge network,** In this context, medical 
schools, professional societies end other 
health related organizations mi st partic- 
ipate by providing evaluated substantive 
Input of Kigh quality and pertinence to 
the mafor health proolems of the nation. 
It was clearly demonstrated at the 
and meeting that medical libraries 
avc been In the forefront In the investi- 
gation and Installation of modem ma- 
chine methods to cope with the rising 
flood of materials being published. Li- 
brarians have felt the need to prepare 
themselves to more fully utilize comput- 
ers in documentation and control of do- 
cumentation. It was suggested that the 
librarian lake advantage of computer 
Irilning on his own campus, because 
from the expressions at this meeting it 
became clear that computer center per- 
sonnel arc excellent sources of informa- 



tion and support. In addition, attend- 
ance at computer courses, workshops 
and other outside advanced training 
programs should be encouraged when- 
ever possible. Libraries should seek to 
employ systems and programming per- 
sonnel as early as possible in any com- 
puter planning development. Sharing of 
information, ideas and programs has 
given impetus to the development of 
several important cooperative ventures. 

Although some librarians apparently 
still believe that students, faculty and 
other library users might resist the use 
of computer terminals in their search to 
locate Information, it w-as pointed out 
that these students arc now being early 
h^lroduccd Into Imlrumentalion and 
computer technology In their medical 
school careers. Therefore, the library 
should be considered as another re- 
source to acquaint the student with dif- 
ferent l)’pcs of equipment with which 
he may be Interacting In his future 
work. 

The final recommendation of the pan- 
el was that continued federal private 
Interest and support for Improved medi- 
cal library senice should ue sustained 
and, wherever possible, expanded. How- 
ever, careful review of on-going systems 
development In libraries should be made 
to avoid duplication of effort and thus 
avoid unnecessary expenditures. 

The panel w as impressed that cost 
sharing and prowam snaring is now tak- 
ing place at all levels in the tradition of 
library cooperation which has developed 
over the past century. Although sofl- 
ware and hardware arc becoming easier 
to acquire, more attention should be giv- 
en to the development of the ‘liveware**, 
the people, reeJed for the effective use 
of those systems. 



A PROGRESS REPORT OF COMPUTER ASSISTED 
INSTRUCTION AT THE UNIVERSITY OF 
OKLAHOMA MEDICAL CENTER* 



Panel: ^Vlllia^l G, Harlcs.^, 5., CKairtnan, Thomaa N, Lynn, M.D , Kobert C. Duncan, Ph.D,, 
Eugene D. Jacobsen, M.D.» Jamei A, Cutter, M.D., Arthur W. Nunnery, M.D, 



In ihe fall of 1966' the University of 
Oklahoma Medical Center Computing 
P'acihly initiated implementation of 
computer assisted insi ruction (CAI) as 
an eoucationa! capability for the faculty. 
The first year ‘ivcs used primarily for the 
indoclrinilion of a staff and the explora- 
tion of the pedagogical jx)ssibililics of 
tljc CAI technique. These act i\i ties have 
been prcs iously re^X)rted. 

The first step In a new Installation 
must be to dclinc csomputcr aided in- 
struction from a philosophical point of 
\ic\v. When this is done two very prag- 
matic problems become apparent, 
WHERP: and HOW it fits into the aca- 
demic goals and actisitics of the Univer- 
sity. These questions must be answered 
before any significant implementation 
takes place. This answer requires a tax- 
onomy of medical education. For con* 
vcnicricc we have disided it into four 
major areas: 

1. Graduate education in the medical 
sciences 

2. In-sers ice training in health related 
fields 

3. Undergraduate medical education 
and house staff training 

4. Continuing medical education 

Our plans Include activity in all of 

these areas. The following disciusion 
will concern the v,uious activities of 
CA! in sevxral disciplines and reflects 
the interest in the four areas mentioned 
above. Only information concerning 

•Pfrrfnfed hyi /dcu?f|f rnftnhfr# 



material presently developed and opera- 
tional is discussed. 

The use of CA! in graduate education 
in the medical sciences bad its begin- 
ning in a course called ‘‘Mcdicai Back- 
grounds.'* The course was developed 
three years ago to accommodate the in- 
creasing enrollment in the graduate divi- 
sion oI the Department of Preventive 
Medicine and Public Health. Students 
entering the disciplines of environmen- 
tal health, health administration and 
biostatistics needed additional back- 
ground in order to converse with physl- 
dans and other persons in the neallh 
professions. The course, “Xfedical Back- 
grounds," was therefore designed prima- 
rily to facilitate communications be- 
tween these students and others in the 
health profession. The course evolved 
into a two-hour didactic course lasting 
one semester. 

The IBM 1401 Data Processing Sys- 
tem was restructured to permit imple- 
mentation of computer assisted instruc- 
tion utrliring four 1050 l)|x: writer termi- 
nals. It was decided that ^Medical Back- 
grounds" was appropriate to augment 
the instruction by means of this educa- 
tional technique. Ten lessons were de- 
veloped utilising the drill approach to 
reinforce the materia! corned by tradi- 
tional tedure. Participation in the com- 
puter assisted instruction portion of the 
course was voluntary, and the student 
reactions were mixed. Some students 

fn ccmputcf XiuUfcd insifMd\cn. 
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found It fascinating; others resented it. 
Many mechanical failures such as the 
paper getting jammed in the typewTiter 
terminal and programming systems fail- 
ures were constant sources of frustration 
to the students. 

The following year the IBM 1500 In- 
structional System was available for the 
reinforcement drill sessions. This elimi- 
nated many of the machine problems of 
the first year and proved to be a much 
more effective presentation. The greater 
flexibility in presentation afforded by 
the cathode-ray tube displays made the 
whole effort a much more palatable one 
for the students. 

There was an effort at evaluation of 
the results during the second year. The 
following obscn'alions were made con- 
cerning grade distributions and their re- 
lafionsnip to the numlvor of CAI lessons 
taken: 



Grade 



Number of Average CA! 
Students licsscns Taken 



A 10 8.8 

D 24 6.45 

C 10 5.5 

n 1 2,0 



Perhaps the most plavtsiblc explana- 
tion for this sup|X)rtivc data is that the 
A students were the more highly moti- 
vated students who can be expected to 
avail themselves of any coucational 
nnlirials available. 

The following is a list of the drill ses- 
sions available to the students in Medi- 
cal Backgrovinds: 1. Medical Terminolo- 
gy'; 2. Cellular Structure; 3. Dermatolo- 
gy; 4. Muscular-Skeletal Sy-stem; 5, 
liematolog)'; 6. Respiratory S)*stcm; 7. 
Kndocrinc System; 8. Cardiovascular 
System; 9. Gastrointestinal Tract; 10. 
ITogenital System. 

There is a lesson presently under de- 
velopment concerned ssith the spinal 



cord, ft is a tutorial approach to the 
teaching of the basic components and 
functions of the spinal cora, and is de- 
signed to replace c lecture as opposed to 
the previous ten which serve as supple- 
mentary drill sessions for material pre- 
sented oy lecture. 

One of the more promising areas of 
computer assisted instruction seems to 
exist when a student is allowed to pro- 
ject his existing knowledge of a subject 
into a simulated situation. This area is 
relatively unexplored for many reasons. 
Currently it is technically difficult to in- 
terface effectively the problem solWng 
and tutorial techniques in the C,M envi- 
ronment. Another difficulty Is the inabil- 
ity of the author to adequately define 
the pedagogy involved in such a tech- 
nique which results in dangerously ne- 
bulous goals for the session. One further 
important problem that inhibits the 
progress of such development is the to- 
tal involvement rt*quirca of the profes- 
sor during the development of the ses- 
sion. 

In an attempt to transcend these 
problems and provide an effective learn- 
ing loo!, the Department of Biostatistics 
and Epidemiology of the School of 
Health Is responsible for preparing a les- 
son In sampling theory Involving a dy- 
namic model of a population. The lesson 
rcq\iircs an interface octween the Coursc- 
vvTdcr language for a tutorial mode 
ar.d the MAT language for a problem 
solving mode. The simulated situation Is 
developed for the student in the Course- 
WTitCT mode. He Is allowed to pick what 
he considers to he the appropriate ana- 
lyiical methods needed to soKx the sam- 
pling problem presented. Once this Is 
accomplished he Is directed to switch to 
the MAT mode. Utilizing Monte Carlo 
methodi fn this problem solving mode, a 
sample Is generated. The student*! pro- 
cedural choice In phase one I* used, and 
the appropriate subroutines arc called 
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by the student to analyze the generated 
data, \^^en the analysis is complete, the 
student switches back to the Course- 
uTilcr mode where structured interaction 
occius between the student and the 
computer concerning the appropriate- 
ness of the student’s anafysis and 
other aspects of the problem. This is 
the critique phase of the student’s prob- 
lem solWng experience and the tutorial 
mode will again be in effect. If the stu- 
dent choose the appropriate methods, he 
Hill be reinforced. If his chosen rnelbo- 
dology was inappropriate, he will be 
told so and a proper method suggested. 
The problem may then be extendSi or a 
“cw problem Introduced and a branch is 
executed back to phase one for another 
choice of methodology by the student. 
The plans are for various data to be cop- 
ied by the student and brought back for 
class discussion. This allows fo; many 
different sampling situations to be pre- 
sented and facilitates class discussion. 

It is important that the students ap- 
preciate the d)Tjamic nature of the situa- 
tion. They must not fed that the situa- 
tion is so structured that there is a single 
solution to any problem presented. Tnis 
system gives him the feeling of problem 
soKing in real limei a live situation 
rather than guessing with the machine 
or the professor concerning the flnsNVcr 
to a structured problem. 

These techniques may be utilized in 
other areas of medicine such as diagnos- 
tic procedures, patient management, 
computation of water and elecirol)ie 
balances, drug therapy and the compli- 
cations of misuse of drugs, and possibly 
in epidemiology concemiug the investi- 
gation of contagious disease models. 

It is NTiy difficult iTKlecd to measure 
the efficiency of such an cduealional ap- 
proach and, as has been slated earlier, 
the total ins'olvemenl required of a pro- 
fessor is an inhibiting factor to this ap- 
proach. But the knowledge that one Ss 



teaching things that are difficult to 
teach under the current sy.'tem Is per- 
haps justification enough ur the effort. 
\tore H'ork needs to be drne concerning 
technique development Vefore appropri- 
ate e\'aluations and tradaional efficiency 
measures can be made. 

The Department of Pediatrics has 
elected to use the tutorial approach in 
programming lessons concerning the 
principles of feeding In infant nutrition 
for junior and senior medical students. 
Under the current curriculum the third 
year students rotate in groups of 12 or 
13 for five weeks on the pediatrics serv- 
ice. It is during this period of time that 
they arc exposed to the computer assist- 
ed instruction material covering infant 
nutrition. The fourth year medical stu- 
dents rotate in groups of two for one 
^vcck In the newborn service. These stu- 
dents are responsible for talking \rith 
the mothers of the newborn w^th refer- 
ence to the feeding requirements, selec- 
tion of the formula, and explaining the 
^propriate preparation proccaures. 
The computer assisted instruction les- 
sons on formula feeding and Infant nu- 
trition arc used as orientation sessions 
for those students. The decision has 
l>cen made to pursue Infant nutrition in 
greater depth utilizing CAI techniques, 
Information concerning molecular nutri- 
tion taught by the Department of Bio- 
chemistry will soon be available utiliz- 
ing the cemputer assisted instruction fa- 
cility. Ultimately, it !s cnrisloncd that 
by the time a freshman medical class 
reaches the third year, all of the infor- 
mation concerning these matters related 
to infant nutrition In the basic sciences 
will be available to the students for 
reference or as refresher courses. Hope- 
fully this will help to bridge a gap be- 
tween the basic science education and 
the clinical experience. 

Generally, at this point, the accept- 
ance by the students involved Is favora- 
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bJe. Previously a Iwo-hour-a-week tuto- 
rial session was required in infant nutri- 
tion by the Pediatrics instructors vilh 
the fourth year students. This has been 
cut to about ten minutes a week. 

One of several unique features added 
to the course in medical physiology for 
the academic year 1963 is the use of 
computer assisted Instruction to teach 
hvo lessons. The lessons coiicem **the 
action potcnllar and “panaealic secre- 
tion.** llic Physiology Department has 
moved cautiously into the area, carefully 
building In evaluation procedures for 
academic effectiveness. For the material 
concerning the action potential, one half 
of the class is randomly selected to take 
the lesson on the computer; the oilier 
half of the class takes the Icssoj by tra- 
ditional lecture. Some weeks later for 
the material on pancreatic scCTction the 
two halves of the class are reversed so 
that the group that had the first com- 
uler lesson now gets the lecture by ti a- 
itional means and the other group gets 
the compulcrircd material. Each half of 
the class wll be tested by the same ex- 
amination. This type of design Nvill ob- 
x-iously allow the students to be their 
own controls and a comparison of the 
lecture versus the compulcrircd material 
can be made for the entire class. 

One of the most attractive things 
about CAl to the Physiology Depart- 
ment concerns the available literature 
today which seems to indicate that the 
less cudo\\ed students receive the most 
l>cncfil from the technique. The stu- 
dents at the top of the class do about as 
well regardless of the method of mate- 
rial presentation, ft is the feeling of the 
Physiology Department that the com- 
puter could, at the very least, provide a 
supplement to the traditional education- 
al $v*slcm which would he helpful to the 
students having difficulty with the tradi- 
tional $)*slcm. 

In the f^cparlmcnt of Ancslhcsiologv* 



there is a lesson developed from a paj>cr 
authored by Dr. Joseph M. White and 
Dr. James A. Cutter concerning resusci- 
tation and anesthesia In crushed chest 
patients. This is an approach to the use 
of computer assisted Instruction In the 
continuing education of physicians. l*he 
materia! is In a tutorial mode and ex- 
plores the latest techniques In emergen- 
cy cases concerning resuscitation and 
anesthesia. The method of Implementing 
this lesson involved the direct program- 
ming of a paper which appears in the 
JOURNAL OF AMERICAN SURGERY 
entitled ‘'Resuscitation and Anesthesia 
in Crushed Chest Patients**. The lesson 
has been presented to a group of 10 gen- 
eral practitioners who attended a (\vx>- 
week course at the University of Okla- 
homa Medical Center in postgraduate 
clinical anesthesia. Incorporated in the 
course is the possibility of selecting any 
area of the subject the student wishes to 
review by simply t)ping "outline". This 
causes the outline of the material cov- 
ered in the lesson to appear on the 
screen. This capability Is available to the 
student at any point during the lesson. 
The lesson has also been programmed 
for the word "Comment" to be lecog- 
nized by the computer anvlimc it is en- 
tered by the student. The screen Is 
immediately cleared and the student Is 
allowed to l)*pe whatever he likes. Ills 
message is recorded on magnetic tape 
and the program suhscqiicntl)* branches 
hack to the point rcaehcti In the lesson 
when **Commcnl" was requested. This 
gives the student the ability to disagree 
with any portion of the lesson or to 
n\alco comments to the author at fny 
time. There arc strong Indicators at this 
point that computer assisted inslrviclion 
will be a significant factor in the future 
of tonlinviing education of physicians. 
Many of the traditional problems, such 
as aislancc of the ph)'sidan from a 
learning center and lack of lime for 
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extensive blocks of continuing education 
effort, are transcended by the very na- 
ture of computer assisted instruction. It 
is entirely conceivable and practical at 
this pJint for a terminal to be placed in 
a remote station and tied by voice grade 
telephone lines to the computing facility 
ill a learning center such as the Univer- 
sity of Oklahoma Medical Center. 

Certainly one of the requirements of 
the successful development of a comput- 
er assisted instruction effort is that the 
faculty understand and want the facili- 
ty. These requirements are met at the 
University of Oklahoma Medical Center. 
The faculty is not only interested but 
they arc also permissive in allowing the 
olhci disciplines (computer theorists and 
learning theorists) to develop their tech- 
niques which arc fundamental to an ef- 
fective system. An effective marriage 
must exist between the computer scien- 
tists, the learning theorists and the fac- 
ulty. Real success can ultimately be real- 
ized only when these individual efforts 
arc intepated In an environment of effi- 
cient and effective learning for the stu- 
dent. 

Summary 

Tlje follmving is a summary of the 
CAI material made available to the con- 
ferees during their visit to the Universi- 
ty of Oklahoma Medical Center j 

1. THE CARDIOVASCULAR SVS- 
TEM (School of Health) — A drill 
session concerning basic cardiology. 
This is an excerpt from the lesson on 
the cardiovascular system presented 
to the graduate class In Medical 
Backgrounds taught by Dr. Thomas 
N. LvTin. 

2. THE HUMAN SPINAL CORD 
(School Cff Health) - A tutorial ap- 
proach to the teaching of the basic 
components and functions of the hu- 
man spinal cord. This is an excerpt 



from a lesson by Dr. Thomas N. Lynn 
for the Medical Backgrounds class. 

3. INFANT NUTRITION (Pediatrics) 
— Two lessons concerning feeding 
principles and infant nutrition in a 
tutorial mode were prepared by Dr. 
Arthur \V. Nunnery of the Depart- 
ment of Pediatrics. One lesson dis- 
cusses breast feeding and the other, 
formula preparation. The material Is 
used by junior and senior medical 
students. 

4 . ACTION POTENTIAL (Physiolo- 
gy) — A tutorial exposition concern- 
ing the action potential in Physiology. 
This lesson is being used by /reshman 
medical students. The professor is Dr. 
Roger Thios of the Deportment of 
Physiology. 

5. PANCREATIC SECRETION (Phy- 
siology) — A tutorial approach to the 
teaching of pancreatic secretion by 
Dr. Eugene D. Jacobson of the De- 
partment of Physiology. Ihis lesson is 
Is being used by freshman medical 
students. 

6. SAMPLING THEORY (Riostatis- 
tics) — This lesson Is an integrated 
approach utilizing tutorial and prob- 
lem solving techniques In computer 
aided instruction. The profc.ssor is 
Dr. Robert C. Duncan of the Biosla- 
tistics and Epidemiology Depart- 
ment. The lesson is to Lc used by 
graduate students in the Department 
of Bioslalistic^ and Epidemiology. 

7. RESUSCITATION IN ANP:STHE- 
S!A AND CRUSHED CHEST PA- 
TIENTS (Anesthesiology) ~ This 
lesson is designed to explore the pos- 
sibilities of the use of CAI techniques 
in research reporting. The material 
was authored by Dr. Joseph M. 
Wlute and Dr. James A. Cutter. It is 
envisioned as an approach to the use 
of computer aided instruction In the 
continuing education of physiciaru. 
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CLOSING COMMENTS 



This conference has been concerned with the role of computers in medical 
education. Past and present cjcpcricnccs of knowledgeable leaders have been 
presented and discussed as well as new theories and ideas. Plans for the future 
solution of problems and expansion into operational services to education have 
been expressed, and it appears that the tools arc available to meet the challenges 
presented here. Our ability and desire to do the job remain to be garnered. 

Two resolutions were presented and endorsed by those in attendance, and 
they are reproduced here. 

1. The heart of this conference has been mlcraclion among knowledgeable 
people. It was the first of its type In this respect and dealt with an im- 
portant topic, computer aided instruction in medical education. Be it 
resolved that a conrcrcnce utilizing the Interactive format and discussing 
computers fn medical education be held on an annual basis." 

2. "Be it resolved that we commend the University of Oklahoma Medical 
Center and the United Stales Public Health Sers’icc for their conduct and 
support of this conference and in parlkular that wc commend Dr. James 
L. Dennis, Dr. Joseph M. Wlulc, l>r. Edward I;. Brandi, Jr., Mr. W'illiam 
Harless and Miss Elizabeth Ray of the University of Okfalioma Medical 
Center, and Dr. Frank McKcc, Mr. Norman Tucker and Mr. Norman 
Rydiand of the United States Public Health Service and the most ef- 
ficient s\ipporting staff." 

The conference was adjourned at noon on Friday, April 5, 1968, 
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